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ABSTRACT 



Method and apparatus for forming color images on 
paper using a soblimation type thennal transfer full- 
color printer using three color inks of yellow, magenta, 
and cyan. A color input signal to be printed is subjected 
to a first color correction process. The first color cor- 
rection process performs a color correction u^ng linear 
and non-linear mflsVitig operations and produces first 
ink-density signals. The first ink-den^ signals are 
judged for reproducibility by the printer by determm- 
ing whether die signals fit within a gamut of colors. If 
judged to be reproducible, the first ink-density signals 
are passed on to the printer. However, if the first ink- 
density dgnals are judged to be out^e the gamut of 
colors, the signals are subjected to a second correction 
process. The second color correcdon process fixst ob- 
tains a target color for color reproduction by the 
printer. Next, the second process sets ink-density sig- 
nals, obtains predicted colors realized by the ink-density 
signals, chooses an output ink-density signal such that 
an evaluation value calculated by the target color and 
the predicted color becomes mtrrimal, and produces a 
second ink-*density signal diat results in optimal ocdors 
within the range of colors reproducible by the printer. 
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MEraOD AND APPARATUS FOR FORMING 
COLOR IMAGES BY €X)NVERIING A COLOR 
SIGNAL TO A FURTHER COLOR DENSITY 

SIC^AL 5 

BACKGROUND OF THE INVENTION 

1. Fkid of the Invention 

The present invention relates to method and apf^ra- 
tus for forming color images on paper with color inks 
by color printer, color copy machine or the like. 

2. Desoiption of the Related Art 

In color CRTs (cathode ray tubes), which are widely 
used for color tekvisicm se^ computer monitors and 
the like, color reproducticMi is made by ccHitroIling the 
luminances of the primary colors of colored light, red 
(RX green (GX and bine (B), based on the principle of 
additive color mixture. On the other hand, color repro- 
duction in hard copying soch as by color printer or ^ 
color copy machine is made by controlling reflectances 
of colored lights with color inks of cyan (Q, magenta 
(MX and yellow (YX which are complementary colors 
of the primaries of colored light, based oa the piind^e 
of snbtractive color mixtore. 25 

However, the q>ectral absorption characteristic of a 
color ink is Isoad, as shown m FIGS. 15(aX 15(6) and 
15(c) which show the spectral absorption characteristics 
of inks used in a sublimation type thermal transfer 
printer, and each rale absorbs mmeoessaxy componeiits ^ 
other than the complementary colors of its reflected 
hght, so that it does not function as an ideal absorption 
fiher. Therefore, processing called color correction is 
performed to reproduce desired colors. 

A method called masking has been enq>loyed for 35 
color coriection in hard copying. Among the masking 
methods, most often used is a method called linear 
masking, which determines an ink-density signal (Y, M, 
Q by qjplying a linear operation to a primary color 
density agnal(Dj2,DGvD^X^vfaich expresses the densi- 40 
ties of complementary colors of a primary color signal 
(R, G, BX as shown in the foUowmg (IX 




0) 

45 



The linear masking assumes that the additive law of 
densities (Lambert-Beer law) holds in color reproduc- 
tion that uses real inks and that color reproduction is 
realized by a linear operation in a whole color ^pace. 
However, it is known that in color rq>roduction that 
uses real inks, eg. by a sut^irnation type thermal transfer 
printer, the additive and p ropor ti onal laws do not hold 
owing to various nonlinear fectors such as resublima- 
tion of inks and internal reflection of inks. 

TherefcMe, there has been proposed a nonlinear mask- 
ing method that determines the density signal (Y, M, Q ^ 
of inks by a high-degree polynomiaL Following is the 
formula for quadratic masking, which is the simplest 
nonlinear ma^dng. 

C = aoDR + aiDg + oiDs + ajZ>ij2 + a^Df^ + 0) 65 

Af = a9DR + aioDg + ouDb + otiPj^ + oiyDc^ + 



-continued 

The quadratic masking performs color correction by 
the quadratic equations incorporating nonlinear factors 
m color reproduction, and the 27 correction coefficients 
aoto a26 are determined by the least square method with 
respect to diflerences of color densities. (See for exam- 
ple, Image processing for color reproduction, Tmagmg 
Part 1, Shashm Kogyo BessatsuX 

Further, color reproductioa for hard copying has a 
problem of the rq>roductble cxAor gamut The range of 
densities realized by a color printer is limited to the 
maximum printing density inherent to each printer and 
the density of the p£^>er used for reproduction. Owing 
to diis limitation on die printable doisities and the exis- 
tence of unnecessarily absorbed components, the range 
of reproducible colors is limifPiri^ so that the color gamut 
rq)roducible by a color printer is generally smaller than 
that by a CRT, whidi employs the principle of additive 
color mixture. 

FIG. 16 shows the color gamuts reproducible by a 
CRT and a color printer in the percq>tually uniform 
CIE L*u*v* color space. FIG. 16 (j) shows the projec- 
tion of the color reproduction gamuts on the u*v* 
plane, FIG. 16 (6) shows the projection <m the L*u* 
plane, and FIG. 16(e) shows the projection on the L*v* 
plane In FIG. 16, the color reproduction range of a 
printer is of the printer whose spectral absorption char- 
acteristics are shown in FIG. 15, and the color repro- 
duction gamut of a CRT b of a CRT in die NTSC 
system. 

Since the color reproduction gamut of a printer is 
smaller than that of a CRT m this way, a s^mal that 
requests a color outside the color reproduction gamut 
of a printer is sometimes iiq>ut to the printer. This case 
happens when at least one of the den^ values (Y, M, 
C) determined by linear masking or nonlmear masking 
described above is less than the density of the paper or 
greater than the maxnnum density of Uie printer. Prior 
arts have performed printing for this nnreproduceable 
ink density using fimiters that set the den^ty value to 
the density of the pq>er if a density less than the density 
of the paper is requested and sets the value to the den- 
sity of the maximum density of the printer if a density 
greater than the mazimnm density of the printer is re- 
quested. 

However, since a density signal of inks and colors 
people perceive are in nonlinear relaticms, the limiters 
for an ink-denaty signal do not provide optimal adjust- 
ment FIG. 17 shows examples of color reproduction 
when limiters are used for an ink-density signal obtained 
by a madcing QpcoiJdtm. In FIG. 17, i=^lf 2, 3, are 
desired colors indicated by input signals, and i= 1, 2» 
3, are the corresponding colors indicated by the limit- 
ers. 

Alternatively, there has been proposed a color cor- 
rection circuit comprising means for performing a pri- 
mary correction operation for correcting an ou^ut 
signal (ink density signals) to have a proportional rela- 
tionship with an input signal (primary color density 
signals), means for performing a secondary correction 
operation for minimizing a difference between the den- 
sity of an input image and that of an output image using 
the least square method and means for selecting the 
result obtained by the secondary correction (^)eration if 
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a difference between the result and the input signal is M2, C2) if the input agnal indicates a color onre- 

smaller than a predetermined value and, if not, the result producible by the printer. 

obtained by the primary correction operation (See JP-A It is to be noted that the term "color density signal'* 

SHO 6^151263). indicates a signal for controlling densities of yellow, 

Also, there has been proposed a color image process- 5 magenta and cyan materials used in a color image form- 

mg ^)paratus in which there is provided a look-up table jng apparatus. Namely, it mdicates an color density 

storing data for judging whether or not an input signal signal in the case that yeDow, magenta and cyan inks are 

is within the color gamut reproducible by the output used and, in the case that yellow, magenta and cyan 

apparatus and, if judged that the mptit ^gnsl is within toners are used, it m<^^t^ a toner-density signal, 

the color gamut, by referring to the look-up table, out- 10 Further, in order to achieve the aforementioned ob- 
puts from a density conversion table are selected and, if jective, the api>aratus of forming color images in accor- 

judged that it is out of the color gamut, outputs from ^^oice with the present invention converts a representa- 

logarithmic conversion circuit and masking ctrcnit (See jivc point Tnrfif^rf>H by upper bits of each input signal 

into a first color density signal (Yl, Ml. CI) by a first 

Thus, accordmg to these prior arts, the ^ 15 cc^or correction operation that performs optimal color 

signals are determmed by a maskmgoperati^ correction for colors reproducible by the printer, 

mput sigmd IS out of the color gamut reproducible by a representative point indicates a 

P™*^* . , ^ * . ... color reproducible by the printer, converts the repre- 

•^'''^r^;!^.^.??.^^ ^! ""^""^ n tentative point that hidicates a color unreproduceable 

SrS^CS^^^^t^f M2, C2) by a second color correction operation, and 

Aeoote reproduction^utofth^^ foDowing the above judgment, determines the color 

SSeX? tTS!^^ densSdgnal (Y3, Msfcg^the above representative 

an^i^ore,necessitatmgtosubj«^ ^ point to be the first color density signal (Yl, Ml, CI) if 

ftb considerable to set correction coefficients for the ^ '^""^ ^ ^ indicate a color reproducible by the 
masking operation so that the ink density signals repro^ ^ ' 

duci^lebytheprintermaybeobtainedevcmto^e^^^ M3, O) of the above repr«^tative pomt to be the 
signal out of the color gamut reproducible by the ^^nal (Y2, M2, C2) if the mput 

printer. However, in this method, a very krgecompre^ 30 afalmdicatesaoolor mmsproduceableby thepn^^ 
sionofacolorspaccispcrformedbythemaskingopcra. Bf„^PP^^^ * composed of a 

tion and, therefore, color reproduction may be made ROM or RAM that stores the above color density sig- 
usfaig colors very different from input colors resulting in ^ 0^3, M3, C3X an address means that receives upper 
an unnatural color reproduction. ^ ^S'^ to be printed and generates an 

As shown in the above examples, when d^red colors 35 address to be given to the above memory nieans, and an 
corresponding to an input color signal are outside the interpolation means that performs an interpolation op- 
coJor gamut leprodndble by a printer, prior arts some- eration uang the color density signal (Y3, M3, C3) 
times perform color r^roduction using greatly differ- output from the above memory means and lower bits of 
ent colors from the ones that people feel to be better, ^ ^V^^ signals to be printed and determines the color 
even the better colors exist withm the «)lor gamut 40 density signal (Y, M, C) corresponding to the ii^jut 
reproducible by the printer. signal to be printed. Fmally, the apparatus performs 

color printing using the color density signal (Y, M, Q. 
SUMMARY OF THE INVENTION The method of forming color images in accordance 

The object of the present invention is therefore to with the present invention performs color reproduction 
provide a method and apparatus of forming color im- 45 Qsing ^ optimal color for even an input signal that 
ages of which people do not feel the degradation of indicates a color unreproduceable by a printer, so that 
image quality by performing color reproduction using method greatly improves image quality. This effect 

optimal colors within the range of colors reproducible ^ due to the fact that the method uses the first color 
by a printer, if a color signal that can not be realized by density signal for an input signal that indicates a color 
the printer is given. SO reproducible by a printer and uses the second color 

In order to achieve the aforementioned objective, the density signal for an input signal that indicates a color 
method of forming color images in accordance with the unreproduceable by a printer. 

present invention comprises a first color correction Further, the apparatus of forming color images in 
process that performs optimal color correction for col- accordance with the present invention performs the 
ors reproducible by a printer and converts an input 55 determination of the color density signal by real-time 
color signal into a first color density signal (Yl, Ml, processing even for an input signal that mdicates a color 
CI), a judgment process that judges whether the input unreproduceable by a printer. The apparatus performs 
signal mdicate a color reproducible by the printer, a color printing of a fiill color nnage having high image 
second color correction process that converts the mput quality for input signals that indicate colors including 
signal judged to indicate a color unreproduceable by 60 those unreproduceable by a printer. 

SSTjL^ilS^ir^pSjL^^ BRIEF DESCRIPnON OF THE DRAWINGS 

color withm the range of colors reproducible by the These and other objects and features of the present 

printer. The method then controls ink densities and invention will become clear from the following descrip- 
forms color images following the judgment process, 65 tion taken in conjunction with the preferred embodi- 

using the first color density signal (Yl, Ml, CI) if the ments thereof with reference to the accompanying 

input signal indicates a color reproducible by the drawings throughout which like parts are indicated by 

printer, and using the second color density signal (Y2, like reference numerals, and in which: 
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FIG. 1 shows a flowchart of the method of fonnmg type thenna] transfer full-color printer using three color - 

color images in the first embodiment in accordaiu::e with inks of yellow, magenta, and cyan, 
the present invention; A block diagram of an apparatus used in the first 

FIG. X shows a block diagram of an image forming embodiment is shown in FIG. 2. In FIG. 2, reference 

apparatus used in the first embodiment; 5 numeral 201 denotes a CPU that performs the method 

FIG. 3 ^ws a detailed flowchart of the judgment forming color images in accordance with the present 

process in the first embodiment; FIG. 4(a) shows a invention. 202 denotes a RAM that is used for a work 

gchemarig color gamut reproducible by a printer in the area for CPU 201 in executing a program. 203 denotes a 

ink-density ^pace and FIGS. 4(6X 4(c) and 4(d) are pro- ROM that is used as storage for the program executed 

jections of the color gamut on Y-C, C-M, and M-Y 10 by CPU 20L 204 denotes an I/O means that reads mput 

planes, respectively; signals (R, G, B) to be printed and outputs mk-denstty 

FIG. 5 shows a detailed flowchart of the second signals (Y, M, Q. 205 denotes a bus that connects each 

color correction process in the first embodiment; of CPU 201, RAM 202, ROM 203, and I/O means 204 

FIGS. 6(a), 6(b) and 6(c) show a result of a color with each other. 206 denotes a controller that controls 

rq>rodnction by the first embodiment with respect to impression energy based on the ink-density signal (Y, 

>i*-v*, fi*-L* and v*-L* planes of a perceptually- M, C) ou^ut fam I/O means 204. 207 denotes a ther- 

nniform color ^kace, respectively; mal head diat impresses an ink sheet, which is not 

FIG. 7 shows a detailed flowchart of the second shown in FIG. 2, with the impression energy controlled 

color correction process in the method of forming color by controller 206 and forms a full color image on an 

images in the second embodiment in accordance with ^ image-receiving sheet of paper, which is not shown in 

the present invention; FIG. 2. 

FIGS. 8(a) and 8(6) show examples of an evaluation The first embodiment realizes the method of forming 

function in the second color correction operation of the color images in accordance with the present invention 

second embodiment, re^>ectively; . _ in the apparatus composed as above by software. A 

FIGS. 9(a), 9(b) and 9(c) are projections oito fi*-v*, flowchart of overaD processing performed by CPU 201 

fi*-L* and v*-L* j^anes for showing a result of an ex- is shown in FIG. 1, and detailed flowcharts of subrou- 

perimental ccicr reproduction by the second embodi- tines of the main procesang are shown in FIGS. 3 and 

ment in a perceptuaHy-uniform color ^>ace, Tcspec- 5. 

tivei^ First, overall procedure is described referring to 

FIG. 10 shows a flowchart of the processmg that FIG. 1. 
creates a table of ink-density ggnafa for input signals in Step 101 inputs an input signal (R, G, B) to be printed 

the method of forming color images in the third em- through I/O means 204. 

bodiment in accordance with the present invention; Step 102 is a first color correction process, which 

FIG. 11(a) shows input signals used for a typical 33 performs optimal color correction for a color reproduce 

neigjUxirtiood opeiaticm in the third embodinieiit, and ible by a printer for an input signal (R,G,B) and obtains 

FIGS. VXb) and 11(c) are those when seen in B-R and a first ink-density signals (Yl, Ml, CI). 
R-<j plaitts, re^pectivdy; Step 103 is a judgment process, which judiges 

FIGS. 12(a) and 12(^) show an es^erimental result of whether the input signal (R, G, B) irKtif a t es a color 
color reproduction by the third embodiment in a per- 40 reproducible by the printer based on the first ink-den- 

ceptually-oniform color space; aty signals (Yl, Ml, CIX If the input signal indicates a 

FIG. 13 shows a block diagram of the ^}parattts of color reproducible by the printer, then Step 103 sets a 

forming color images in the fourth embodiment flag signal fig as flg^O. If not, then Step 103 sets flg= I. 

FIG. 14 shows a method of interpolation by the a^ypa- Step 104 is a conditional branch instructioo. If flg=0, 
ratusoffonning color images in the fourth embodiment; 45 ix. the input signal (R, G, B) i ndi ca t e s a color reproduc- 

FIGS. 15(aX 15(ji) and 15(c) show the spectral ab- ible by the printer, then the prooessng proceeds to Step 

sorptkm characteristics of inks used in a sublimation 106. K flg== 1, Le. the input signal (R, G, B) indicates a 

type thermal transfer printer, req>ectively; color unreproduceable by the printer, then the process^ 

FIGS. t6(a\ 16(b) and 16(c) show color gamuts re- ing proceeds to Step 105. 
producible by a CRT and a printer with re^>ect to 50 Step 105 is a seccmd color correction process, which 
|i*-v*, ;i*.L* and v*-L* planes, re^)ectivel^ performs a second color correction operaticHi for an 

FIGS. n(a), n{b) and 17(c) show color reproduction input signal which can not be realized by the printer and 
by a prior method of forming color images with re^>ect obtains a second ink-denaty signal (Y2, M2, C2) that 
to ^*-v*, ^*-L* and v*-L* planes, in case an input produces an optimal color reproducible by the printer, 
signal indicates a color unreproduceable by a printer. 55 Step 106 outputs an ink-density signal (Y, M, C) from 
^^.wv^^v^o^^T^^^^T^^-v^TT^ I/O means 204 dependmg on the flag signal flg. If 

^^I^S^J^^^^^I^^^ flg=0. then (Y. M, qI^I, Ml, CI), and if flg= 1, 

PREFERRED EMBODIMENTS tlL (Y, M, C)=(y2, M2, C2). 

The preferred embodiments according to the present Thai oontroUer 206 controls the heat quantities of 
invention will be described bdow with reference to the 60 thermal head 207 in the face order of yellow, magenta, 
attached drawings. and cyan depending on the ink-<lensity signal (Y, M, C) 

In the following is described the method of forming output from I/O means 204, so that gradation reproduc- 
color images in the first embodiment in accordance with tion is performed on an image-receiving sheet of paper 
the present invention. The first embodiment determines and a color image is formed. 

an ink-density signal (Y, M, Q by software for printing 65 Next, the first color correction process performed by 
or reproducing color images by a printer based on a Step 102 is described in the following. In *^e present 
primary color luminance signal ^ G, B) that is used for embodiment, a mixed operation composed of linear 
a CRT. The first embodiment employs a sublimation operation for luminance signals and nonlinear masking 
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'operation for doasity' signals^ is used for the fiist color 
correction. The details are as follows. 

Firstt the foUowing luminaix;e matrix operation (3) is 
applied to the input signal (R, G, B) to obtain a second 
Imninance signal (R', G\ BO in order to correct the 
difference of the central wavdengths between the spec- 
tral characteristics of the phosphor used in the CRT and 
the spectral absorption characteristics of the inks used 
in the printer. 



8 



for about color chips uniformly located' in the color 
gamut reproducible by the printer. The actual values of 
these correction coefficients are as fc^ows. 



)i-t 



hi 



0) 



Next, each oonqx>nent of the second luminance signal 
(K', G\ B') is converted into a component of a primary 
color density signal (Or, Dg, T>b) for subtractive mix- 
ture of colors by the following complementary color 
transformation (4). 

D/i^logjoCl/R') 
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I>C='IoglO(l/G') 
DF=logio(l/BO 



(4) 
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Further a nonlinear masking operation is applied to 
correct color turtndity caused by components of light 
unneoessarily absorbed by inks. 3q 

First, the primary color density signal is converted 
into a agnal corresponding to color materials of mks. 
Specifically, if a constant that eipresses the degree of 
nonlinearity in the relation between the color matfrials 
of inks and the color densities is denoted by a, where a 35 
> 1, then, by a first nonlinear transformation formula 
shown m the following (5), the primary color density 
signal (Pr, Dg» Db) is converted into a signal (C, M', 
Y*) oortesponding to color materials of inks. 



40 



(5) 



Next, by a linear transformation shown m the follow- 
ing (6), the output signal (C, M', V) of the first nonlin- 
ear transformation is converted into (C, M", Y"). 



45 




(6)50 



Further, (C", M", Y") is converted into (Yl, Ml, CI) 
by the following (7), which is the inverse of (5). 

In the present embodiment, the correction coeffici- 
ents {bjki} of (3), {ajki} of (6), where, k=l, 2, 3, and 65 
1 = 1, 2, 3, and the constant a of (5) and (7) are deter- 
mined by a successive approxintiation so that the aver- 
age errors between each color reproduced by a CRT 
and the corresponding color reproduced by the printer 



■frll b\2 Al3 
^21*22*23 

^1 *n *J3 



1-E 



a896 a096 aoo8 
0.002 aa3S 0163 

a068 -0035 0967 



(8) 
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1.443 -0.105 0.037 
-0.360 1.694 -0032 
0.002 -0.679 1.978 



a = 1.572 

The first color correction operation used in the pres- 
ent embodiment performs color correction for colors 
reproducible by the printer with the color diffisrence 
Euv=4.3 on an average in the perceptuaUy-uniform 
color space. 

Next, the judgment process (Step 103) that judges 
whether or not an input signal (R, G, B) indicates a 
color reproducible by the printer is described in detail 
in the following. 

FIG. 4 shows the color gamut reproducible by the 
printer in the ink-density spstcc, which expresses the 
den^ of each ink of yellow (Y), magenta (M), and 
cyan (Q by a coordinate on the corresponding axis. 
FIG. 4 (a) shows a perspective view of the color gamut 
FIG. 4 (p) shows the color range projected on the YC 
plane. FIG. 4 (c) shows the color range projected on the 
MC plane. FIG. 4 (d) shows the color range projected 
on the YM plane. As shown in FIGS. 4(a) to Md), the 
color gamut is determined by the density that can be 
reproduced by each ink. If the color densities of the 
paper used for printmg of yellow, magenta, and cyan 
are respectively denoted by YO, MO. CO, and the corre- 
spcHuling maximum densities that can be printed are 
respectively denoted by Ymax, Mmax, Cmax, then the 
color range is the rectangular solid enclosed by the six 
planes, Y=YO, M=MO, C=CO, Y=Ymax, M=Mmax. 
C=Cmax. 

Therefore, when a first color correction operation 
performs optimal color correction for colors reproduc- 
ible by the printer, whether the input signal (R, G, B) 
indicates a color reproducible by the printer is judged 
by testing whether YO^ Yl^ Ymax, MO^Ml^Mmax, 
and CO^Cl^Cmax for the first ink-density signal (Yl, 
Ml, CI) obtained by the first color correction process. 
Step 103 performs this test The detail is described in the 
following referring to the flowchart of FIG. 3. 

Step 301 tests whether YO^Yl^ Ymax or not If not 
then the input signal is judged to indicate a color unre- 
produceable by the printer. Similariy, step 302 tests 
whether MO^Ml^Mmax or not, and Step 303 tests 
whether CO^Cl^Cmax or not 

If the input signal is judged to indicate a color unre- 
produceable by the printer by one of Steps 301, 302, and 
303, then Step 305 sets fig= 1 . If the input signal satisfies 
all the test conditions of Steps 301, 302, and 303, then 
Step 304 sets flg=0. 

In this way, the present embodiment judges whether 
the input signal (R, G, B) indicates a color reproducible 
by the printer based on the first ink-density signal (Yl, 
Ml, CI) obtained by the first color correction prxxjess. 



5,448,379 



10 



Next, the second color correction process (Step 106) 
is described in the following. The second color correc- 
tion operaticm first obtains the color indicated by the 
input signal (R, G, B% Le. a target color for cc^or repro- 
dnction by the printer. Then the seocmd color correo- 
tion operation sets ink-density signals, obtains predicted 
colors realized by the ink-density signals, chooses an 
optimal ink-density signal such that an evaluattcm value 
r^lrnlatPd by the target color and the predicted color 



Step 505 judges whether the Euv obtained by Step 
504 is mtntmfll or not 

Step 506 stores the ink-density agnal set by Step 502 
as the second ink-density dgnal (Y2, M2, C2), if Step 
506 judges that the Euv is mtwmiftl 

Step 507 tests if the search for an optimal ink-density 
agnal has been finished or not If the search has not 
been finished, then the processing returns to Step 502, 



becomes minhnaL FIG. 5 shows a detailed flowchart of 10 ^^«Peats Stej^^from 510 to 507, 



the second color correction process. Hrst, Step 501 
converts the inpot signal (R, G, B) into a color signal of 
a target color f^ color reproduction by the printer. In 
the present invention, the input signal is a primary color 
luminance signal G, B) that drives a CRT. There- 
fOTe, tristimulus values (Xo, ZoX which are repro- 
doced when the primary color huninanoe signal is input 
to a CRT of the NTSC system, is obtained by the fol- 
lowing ou^ut equation (9) for a CRT of the NTSC 



060699 
0^97 
00 



ai734S 
058642 
0066075 



020057 
011461 
1.11586 



Further, considering human visual characteristics, 
the tristimnlns values Yo, Zq) are converted into 
the coor di nates (Lo*, uo^t vo*) in the peix:eptnally nni- 30 
form L^*v* space by the following (10). 



Next, the prediction <^ color reproduction in Step 
503 for a set mk-den^ signal is described in the foI> 
lowing. The first color correction operation in the pres- 
ent embodiment is a nonlinear operation including the 
15 nonlinear transformatioiis (4), (5), and (7). Since each of 
these nonlinear transformations and linear transforma- 
tions (3) and (6) is a one-to-one transformation, a con- 
version of the ink-density signal into a primary color 
luminance signal can be obtained by the inverse of the 
^ first color correction operation. Therefore, the present 
embodiment performs die pfedidion of color reproduc- 
(9) tion by converting the ink-density signal into a primary 
color luminance signal and then ccmverting the results 
into the color signal in the perceptually-uniform color 
space in the same way as a target color is obtained. 

Spedfically, theset mk-density signal (C2, M2, Y2)is 
converted into (C2", M2", Y2") by the following non- 
linear transformaticMi (12). 
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Lo* ^ 116(r(/ra^ - 16^ if 7o/7„ > 0008856 
- 903.29(To/7«X 'iTTo/YmS O00S8S6 



(10) 



mi* » I3io*(«o* — »«') 

10' = 13I«*(io' - O 



35 



(12) 



where 



Then (CT', MT'. YZ") is converted into (CT, M2', YZ) 
by the following (13), which is the inverse of transfor- 
mation (6). 



and 



ao'=4io/(X(H- 15Yo+32<i) 
vo'=9Yo/CXo+ 15Ya+3ZQ) 

Y«»100 



40 



45 



[cr "I pii «u owl fcr ] 
Mr = 021 022 0Z3 \*^ ' \ 
^ ] [c3i an a^] J 



(13) 



u«'=0l2009 



v.'bO4609 
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if a standard light source used for illumination is the C 
light source and the field of view is 2 degrees wide. 

Next, Step 502 sets values of an ink-density signal to 
search for an c^timal ink-density signaL 

Step 503 predicts the color (L/», u/*, v/*) reproduced 
by the printer when the ink-^lensity signal set by Step 
502 is used. 

Step 504 obtains the evaluation value Euv for the set 
ink-density signal ussag the target color (Lo** uo*, vo*) 
and the predicted color (L^*, u/*, v/*). The evaluation 
value Euv in the present embodiment is the color differ- 
ence in the perceptually uniform space between the 
target color and the predicted color defined by the 
following (11). 

Eav={(Lo'-Lf')^+(no*-n.*)2+(vo*-vi«)Oi 
(H) 



Then (C2', MZ, YZ^ is converted into a primary color 
luminance signal (P2r, D2g, 02^) by the following 
nonlinear transforation (14). 



55 



(14) 



60 



Further, (PZr, D2g» I>2j) is converted into a lumi- 
nance signal (RZ, GT, BZ) of additive color mixture by 
the inverse of the complementary color transformation 
(4). 



Gr=\<rmG 

65 Br^ltTDlB (15) 

Then (R2', 02*, BZ) is converted into a primary color 
lummance signal (R2, G2, B2) by the following linear 
transformation (IQ, which is the inverse of (3). 
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The prediction for the ink-density signal not listed on 
(16) the table is obtained by interpolation. However, this 
method needs a large number of color chips and a vast 
amount of time for measurement. Moreover, since the 
5 color gamut reproducible by a printer has a complex 
shape m the space of input color signals, errors caused 

FmaUy. (R2, G2. B2) is converted into tristnnulus ^ ^ ^ '^t 

values (Xa Y, Zd by the tramfonnation of (9) and then °^ predK^tmg color reproduction m the pr^t 
converted into coordinates (L,* uA v.') by the tians- ,„ e«bodmient does not n«d time and labor for makmg 

formation of (10) table and obtains about the same precision as the 

As mentiwied before, the first color correction opera- ^ ^^^^ correction operation, 
tion is performed with Euv=4.3 on an average for Next, the second embodiment of the method of form- 
colors reproducible by the printer. The second color ^8 ^^^^ mtages in accordance with the present inven- 
conection operation is also performed with about ^^n is described m the following. The experimental 
Eov=4.3. apparatus used in the second embodiment and the over- 
Results of an e3q;>erimental color reproduction by the ^ flowchart is the same as in the first embodiment, but 
first embodiment are shown in FIG. 6, which i^ots the second color correction process is different, 
target colors and corresponding reproduced colors in The second color correction process in the second 
the perceptually uniform L*u*v* ^»ace. In FIG. 6, Pi „ embodiment is described in the following. FIG. 7 shows 
i=l, 2, 3, are target colors which are unreproduceable a flowchart of the second color correction process of 
by a printer, i= 1^ 2, 3, are corresponding colors the second embodiment The second color correction 
reprtxluced by a prior techniques using a masking oper- operation of the present embodiment varies an evalua- 
ation and limitcrs, and i~ 1» 2, 3» are colors repro- tion fimcdon depending on the input agnal (R, G, B) 
duced by the present embodiment It is observed that and obtains an ink-density signal such that the evalua- 
the color correction of the present embodiment uses a tion value becomes mmtma! 

color reproducible by the printer and locating apart First, Step 701 converts the input signal (R, G, B) into 

from its target color by a minimal di sta n ce, a coordinate (Lo*, uo*, vo*) in the perceptually imiform 

In this way, the present embodiment performs color L*u*v* color space. 

c(»Tection using a color reproducible by a printer and Step 702 determines the evaluation function depend- 
having a minimal distance from its target color m the ^ jng on the mput signal. 

perceptually uniform space to improve image quality. step 703 sets an mk-density signal to search for an 

The conqwMtion of the apparatus, the process of optimal mk-density signal, 

determining the ink^ienaty sigmd for an input sigi^ step 704 predicts the color (L/*, u/*. v/*) reproduced 
Sffo^*^^!^^ ' 35 ^>y the printer if the ink-<lensity signal set by Step 703 is 

Intibe present oi^odiment, thejudgnent on wh^ ^ evaluation value for the set ink- 

an input signal ni^cates a color reprodu^le bya density signals using the target color (Lo*, no*, vo*) and 

prmtcr or not is performed usmg the result of the first *tr^lLi-lI!T,^ 5 ^ I? *i. i 

SIL—*; A J * • the predicted color (L,-*, Ui*, Vi*) based on the evalua- 

«^or correctKm operation. Among methods not uang a^^uU i«, c*L taa 

this result, there is a method that uses a table of Mfi-^*^c.^^-^^^ ^ , »^ 

ment results on whether each of aU possible mpirt sig- ^^J^ judgK whether the evaluation value ob- 

mils is reproducible by a printer or not The Wis tamed by Step 705 is nu^ 

made, for example, by expressing colore repiodudblc mk-densrty signals set by Stq> 703 

by aprintcras a range oftheperceptoaUy-uniform color as the second hxk-denaty signal (Y2, M2, C2). if Step 

space, aid comparing each input signal with this range. 706 judges that the Euv is mmimaL 

Since the color gamut reproducible by a printer has a tests if the search for an optimal ink-density 

complex shape in the space of mput color signals, a vast signal has been finished or not If the search has not 

amount of time is necessary to malcp> the table. More- ^^^en finished, then the processing returns to Step 502, 

over, if an input signal is expressed by 8 bits for CTch and repeats Steps from 703 to 708. 

color, then a large memory of 2^ bits is needed. In 50 The prediction by Step 704 of color reproduction for 

contrast, the judgment method of the present embodi- a set ink-density signal is performed by converting the 

ment requires the comparison operation for the first ink-density signal into a color signal in the L*u*v* color 

ink-density signals (Yl, Ml, CI), so that the processing space in the same way as in the first embodiment 

time is saved when the judgment is performed by soft- Next, the evaluation fiinction that evaluates the opti- 

ware, and the circuit scale is saved when the judgment 53 mality of a color reproduced by a printer in the present 

is implemented by hardware. embodiment is described in the following. 

In the present embodiment the prediction about The evaluation fimction £ is given by the following 
color reproduction for a set mk-density signal is made (17), where AL, AH, and AS respectively express differ- 
by the inverse operation of the first color correction ences of Ughtnesses, hues, and saturations between the 
operation. Among methods of predicting color repro- 60 target color (Lo*, uo*, vo*) and the predicted reproduc- 
duction for a set ink-density signal is a method of using tion color (Li*, Ui*, v/*), and a, b, and c are weights, 
a table showing the correspondence between ink-den- Here, the saturation and hue are those in the L*u*v* 
sity signals and color measurements of color chips made color space, and the difference of hues is multiplied by 
by ink-density signals. The color chips are prei^ued for the average of the saturations of the two colors to equal- 
colors situated in a boundary of the color range repro- 65 ize the units, 
ducible by the printer, i.e. for each ink-density signal 

such that at least one of its values is a color density of E^iiaAL^-^bAH^-i-c^s^ia+b-^e)}^ (I7) 
the used paper or the maximum density of the color. 
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where perceptually uniform space. Wbite is reproduced as a 

color having the density of paper, and black is repro- 
AL =: lo* - V doced as a color having the maximal density. 

_ The process of determining the ink-density signal for 

^ " ^ = So*^ + ^ ^ signal, the method of determining the evalua- 

5/ =3 (o/*2 + tion function depending on input signals, and results of 

experiments have been described above for the second 
ao-tta-W/^)*) embodiment. 

0/ = taii-*(vi*/ui*) In this way, the present embodiment performs color 

correction using colors that are reproducible by a 
The weights a, b, and c are varied depending on input printer and people fed to be most preferable based on 
signals and determined by the target color indicated by evaluati(Hi functions depending on input signals, so that 
dte hxput signals dnongh an experiment of subjective hnage quality is greatly improved, 
c^'**'*^**^'^ . . 15 present embodiment uses evaluation functions 

Rrst, the weights of the evaluation function E are shown in FIG. 8, but since the color gamut i^>roduc- 
detenmned for 26 mpnt agoals that mdicate colors ibie by a printer varies with the printing method of the 
tocated on tte boundanr^^ the range of colors reprt> ^ ^ ^ characteristics of the inks to be 

duciWebya CRT m the color spa^ used, evaluation fanctions vary with printers. TW 

agnals. The target color of each of the 26 signals is i *i «. ^ r *i. ♦ • 

by converting the signal into coordiSin ^ ^"""^ the ev^tKm functKM^ of the p«ent mvention 
the L»u*v* color space. Then three evahiation func- ^ to the ones shown m FIG. «. 

tions, each having a combination of weights, are used to ^he method of forming color nnages of the third 

make three color chips for of 26 input signals such embodiment in accordance with the present invention is 
duit the ink-density signal used for making each cc^or described in the following. 

ch^ minimizes ea^ evaluation function. Then ea ch ^ The apparatus used in the third embodiment is the 
color cfaq> is con^jared with the correspcxiding target same as the one in the first embodiment, but the process 
coIot, and one color chip is chosen as an optimal color of determining the ink-density signals is different In the 
chq> by subjective judgment, so that the evaluation third embodiment, the ink-density signals for input sig- 
fimction for each ofthe 26 input signals is determined to nals are stored in RAM 202 as an LUT Oook-iq> table), 
be die evaluation function that was used to make the ^ and the ink-density signal used for each input signal is 
optimal color chip. The three evaluation functions con- output by i rffn in g to the LUT. The ink-dextsity signals 
sist of a lightness^ ph asi z ed funclicHi, where weight a ^^^^ jjjj ^ determined using a neighborhood opera- 
b set greater than b and c, a hue-emphasized function, tion m the color space of mput signals, 
whcrewei^issetere^^ HG. 10 shows the procedure of determinmg ink-den- 

sitysigmdsfortheLirr. 
the'SJL^^^SSB'^^ TirT^lOOlsets^put^ 
roTSS ^^Ss^^^ ^^iT^^J^Se^ step ob^ thefet ink-density signal (Yl, M 1, 
CRT viewed frOTi white (WX and FIG. shows the CI) as m the first embodment. , ^ ^ 

cube viewed frOTi blade (Bk). Each smaDdrdeiiMiicate^ Step 1003 judges whether the mpot signal (R, G, B) 
the coordmatc o( one of the 26 input signal and its eval- indicates a color reprodudWe by the printer based on 
natkm function. For example, if the target color is blue the first ink-density signal (Yl, Ml, CI). If the input 
(B)oftheCRT, then the uniform function bits evalua- signal indicates a color reproducible by the printer, then 
tioo fisnction. if the target color is green (G) of the Step 1003 sets a flag signal flg as flg=O.Ifnot,dien Step 
CRT, then the lightness-emphasized fimction is its eval- 45 1003 sets flg=l. 

uation function. Ifthe target color b black (Bk) or white St^ 1004 b a conditional branch mstruction. If 
(W), then any one of the three evaluation functions can fig— 0, Le. the input signal (R, G, B) indicates a color 
be used. reproducible by the printer, then the processing pro- 

As shown m FIG. 0, the present embodiment deter- ceeds to Step 1006. If flg= 1, i.e. the input signal (R, G, 
mines the ink-density signal for each input signal using 50 mdicates a color unreproduceable by the printer, 
an evaluation function whose weights are continuously ^ processing proceeds to Step 1005. 

vari^wth input sigMls. , ^ ^ ^ ^ Step 1005 obtains, for an input signal (R, G, B) unre- 

FIG. 9 shows^«nmental results of color reproduc- produccable by the printer, the second ink-dcnsity sig- 

« nal(Y2,M2,C2) that produces an optimal color repro- 

~ ^1 ""f^i^S!?^-^ k'^T^ d«=ft>le by the printer. The process b the same as of the 

green, and blue, which are not reproducible by a pnnter , ^, i ^ ™ ^u^i 

and their reproduced cotersarTZwnm the L^^^^^ correction process m the first embodi- 

space. In FIG. 9, points P4 and R4 respectively show ^ ^ . , j • i /v^ 

the target color and the reproduced coM red; P5 and ^t^^*^ f °^:*^^f ^ 

R5 respectively show the target color and the repro- 60 ^3) from the first mk-density signal (Yl, Ml, CI) and 
duced color of green; P6 and R6 respectively show tiie second denaty signal (Y2, M2. C2) dependmg on 

target color and the reproduced color of blue. Since a the flag signal fig and temporarily stores (Y3, M3, C3) 
hue-emphasized evaluation function b used for the »fl RAM 202. If flg=0, then (Y3, M3, C3)=(Y1, Ml, 
input signal indicating red, red b reproduced as a color CI), and if flg= 1, then (Y3, M3, C3)=(Y2, M2, C2). 
having almost the same hue as itself and reproducible by 65 Step 1007 tests if the third ink-density signals (Y3, 
a printer. Similarly, green b reproduced as a color hav- M3, G3) have been obtained for all input signals. If not, 
ing almost the same tightness, and blue b reproduced as then the procedure returns to 1001; else the procedure 
a color having a minimal distance from itself in the proceeds to Step 1008. 
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From Step 1008 and onward die procedure performs described above for the third embodiment The present 
a neighborhood operation for each third ink-density embodiment can suppress sudden changes in color re- 
signal (Y3, M3, C3). production for continuously dianging target colors to 
Step 1008 sets an input signal reduce discontinuous color reproduction by obtaining 
Step 1009 applies the ndghborhood operation to the 5 an optimal color for each mput signal and applymg a 
third mk-density signal corresponding to the input sig- tbree^iimensional neighborhood operation in the color 
nal set by Step 1008. 

St^mOstored&er^tofS^ pre^t eSiment limi^ the number of input 

^ ^« ^.^"^^ ^ * ^SB^ to 6 to perform andghborhood operation. How- 

Step 1011 tests if the neighborhood operation has lo *™ . . ^ * i* u 

, ^ ... , „ . ^ V ^ ' ^ , *^ Mi/*t i«w IV jjjg present mvention does not limit the number 

been applied for all u^>ut signals. If not, the procedure ^ t.._j • , ^ • . . u ^ j *u «- 

returns to Step 1008rdse thVprocedure terminates. of the thud inkKienaty 

Next, the ndghborhood operation of Step 1009 is "^^^^ ^ ^"^"^ <*^> ^ neighborhood operation 
described in the following. to those of the present embodiment 

The third ink-density signal is chosen from the result 15 Next, the ^paratus of forming color images of die 
of the first color correction operation and the result of fovith embodiment in accordance with the present in- 
the second color correction operation for the mput vention is described in the following. 
signaL The second color correction operation obtains The fourth embodiment uses, as its input signal, a 
an optimal ink-density signal for an input signals unre- Y-corrected luminance signal (r, g, b), each component 
produceable by a printer, so that, owing to the shape of 20 being an 8-bit signal, and forms color images by a subli- 
the color gamut reproducible by the printer, repro- mation type thermal transfer printer using inks of cyan, 
duced colors sometimes suddenly change even if tlie magenta, and yeUow. 

input signals continuously vary. As a result, the smooth in the fourth embodiment, the ink-density signals for 
gradation of a reproduced image b damaged. There- 32 X 32x32 representative points, each expressed by the 
fore, the present embodiment performs a three-dimen- 25 upper 5 bits of each 8-bit component of the input signal 
sional spadsl -filtering in the RGB luminance color 5)^ ^re stored m an LUT (look-up table). The 

^>^by &e ndghborlK)od <^)era^ ink-density signal for a pomt between those representa- 

FIG. 11 ahows an mput si^ und^ conaderation tive points is determined by the method of three^limen- 
and^ghboimg mput sigaab referred m the DCigh- ^ interpolation. 

borhood operatKm m the RGB col^ space. If the Y 30 Before the composition and operation of the present 
component of the Idiiid mk-da^ embodhnent is dSU the meAod of interpolation is 

s^ under consui^^on is toK^ Y3(R, G. B). "^T^ refer^ to FIG. 14. FIgT^ows a 

and the Y component of the third mk-density signals for ^^aouk" wiui i ci»ciu^ ^xxj, * auww» 

the6neighboringhq>utsignalsaredenotedbyW-l» ««« cube formed by neighbonng 8 r^rcsentative pomts 
G, BX Y3(R, G-1. B), Y3(R, G, B-IX Y3(R-hl, G, « ^ • ' • '^."^ *f f ^^f' ^P^^ ^snal 

B% Y3(R. G+1, B), and Y3(R, G, B-flX then the expressed by a pomt P mside the cube, the three planes 
fourth ink-4ensity dgnal Y4^ G, B) for the mput sig- paraDel to rg plane, gb plane, and rb plane passing 
nal under consideration bdetcrmmed by the followmg through P divide the cube mto 8 rectangular solids, and 
neighb(»rhood operation (18). the volume of the rectangular soKd containmg the ver- 

tex Fk b denoted by Vk. If the mk-density signal of the 
^lgj representative point Pk b (Ck, Mk, Yk), then the ink- 
n{,K a iO = (2 X T3iK a J) + tHR - t a jB) + density signal of point P b determined by the following 

7KR, c - t ^ + y3(R a 5 - I) -I- interpolation formula (19). 

n(« + 1, a J) -h iw c + I, J9) -I- 

I3(R a B + I)} X 4 

45 7 (19) 

The M and C components of the fourth ink-density ^"0 
signal are detennined by similar operations. 

FIGS. 12(a) and 12(6) show an experimental result of i Afk- vk/v 

color reproduction by the present embodiment FIG. 12 

(a) shows the range of colors reprodudble by a printer 50 ^ 

m u*v* plane of the L*u*v* color space, and FIG. 12 r = 2 nk . Vk/v 

(b) shows the encircled part in FIG. 12 (a). Target col- 

ors and corresponding colors reproduced by a printer , ^ ^ , 1^ ^. . ^ 

by the experiment are plotted m FIG. 12 {h\ where composition of the fourth embodnnent is de- 

each small circle indicates a target color, which b unre- 55 the foUowing. FIG. 13 shows a block diagram 

produceable by the printer, the trian^e connected to the apparatus of formmg color images m the fourth 

the target color with a dotted line indicates the corre- embodnnent 

spending reproduced color using the third mk-density ^ 1^' 1^1 an address generator tiiat reads the 

signal, and the square coimected to the target color with "^PP^ ^>ts of each component of an input signal (r.g 

a solid line indicates the corresponding reproduced 60 *>) outputs the addresses of the 8 representative 

color using the fourtii ink-den^ signal. As observed points of the unit cube that contains the point expressed 

fixnn FIG. 12 {b\ the color reproduction using the by the input signal in synchronous with the CLK signal 

fourth ink-density signab improves the continuity of of tiie input signaL 

color reproduction, when the color reproduction using 1302 b an LUT memory that outputs an 8-bit tnk-den- 

the third ink-density signab produces unnaturally dis- 65 sity signal (Y, M, Q corresponding to an address output 

continuous color changes. from address generator 1302 and a 2-bit CX>LS£L sig- 

The process of determining the ink-density signal for nal that indicates one of cyan, magenta, and yellow 

an input signal and results of experiments have been currendy being printed. LUT 1302 b configured by a 
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ROM of IM bits (128K bytes) aod uses an area of 96R signal by (3) to (7) of the first color correction process. ^ 

bytes. Then whether the first ink^ensity signal is reproducible 

1303 is a counter that counts up in synchronous with by the printer or not is judged by the procedure shown 
a CLK signal and outputs a 3-hit signal mdicating the k m piQ, 3. if the first ink-Klensity signal is reproducible 
of formula (19) 5 by the printer, then the first ink-density signal is stored 

1304 is a weight table memory that memorizes ^UT memory 1302. Else, the second ink-density 
weights Vk/V of (19X reads the lower three bits of the determined by the procedure shown in HG. 5 
mput signal (r, g, b) and the addras output from ^ operations (9) to (IQ is stored in LUT memory 
counter 1303, and outputs a weight VVV correspond- ^^02 

° mSfa a^S^g means that multiplies each out- , '^.^ conation and operation of theappanitus of 

put^LUT n^^S^y each o^ut of weight ^« ^i^TT "t^' f^^^^Kxhrnent have 

teble memory 1304, adds up the produ^in syn^ ^ describ^tabove. The niethod of formmg color 

noos with die CLK signal to perform the calculaticm of ™^ accordance with the present mvendcm re- 

(19), and ou^ an mk.<lenstty signal (Y, M, Q to be operation time for the second color correction 

used in printing. operation of d^ennining optimal ink-density signals for 

1306 s a controller that controls the impression en- input signak unreproduceable by a prints. Howevo-, 
agy depending on the ink-density signal output from the fourth embod im ent stores the ink-density signals in 
r^lrnlatmg meaus 1305. an LUT memc^ beforehand, so that the ink-density 

1307 is a thermal head that impress an ink sheet with 20 signal is determined by real-time processmg for an input 
heat depending on the impression energy controlled by signal unreproduceable by a printer. Further, memory 
controller 1307 and forms a full color image on a sheet size is smaller than in an apparatus storing the ink-den- 
ofp^^. sity signals for all possible input signals in an LUT mem- 

The operation of the apparatus of the present emfoodi- oxy. As in the first embodiment, color reproduction is 

ment for printing the first color, Le. yellow, is described ^ performed using optimal available colois for input sig- 

m the following. anrq)roduceable by a printer, so that the present 

Rrst, address generator 1301 successively outputs the apparatus greatly improves image quaUty . 

addressesof the 8 vatices of the unit cube correspond- Embodhnents of the method and apparatus forf<wm- 

ingtoankput sigBffll^msyncte^ ing color images in accordance with the present mven- 

f^f^'^'^Sf^^'^?^^^ 30 tion have been described above In each of these em- 

M ^5 Swi^o-^?*?^ ^if 'iS ^'^^^ the mput signal is a lummance signal (R, G, 

^ ^^tl'^i ^If^^'^X (T:^fM^^ B) that drives a CRT, and the first color correction 

outputs the ink-density signals YO, Yl, . . . , Y7 of yd- ^nd nonhnear maskmg opetations. However, the first 
low cooespondmg to tiiese addresses. color correction c^)eration is not hmited to that opera- 

Qn the other hand, the lower 3 Wts of the mput signal example, any color correction operation hav- 

(r, g, b) and tiie ou^ of ccwmter 1303 are provided to inverse soch as a prior Imear masking opmtion 

weight table 1304, winch successively ou^Mits weights ^ ^ ^sed as the first color correction operation, and 
VO/V to V7/V. the color prediction in the second color correction 

Then falrmlflttng means 1305 calculates formula (19) ^ operatkm is performed using the inverse of the first 
imrt ou^Hds the ink-density signal Y of yellow for the color correction operaticHL 

ixq>ut signal (r, g, b). Alternatively, the first color correction operaticm can 

Controller 1306 contrc^ the impression energy de- be performed by a prior nonlinear masking operation, 
pending on the output signal Y of calculating means and the color prediction in the second color correction 
1305, and thermal head 1307 impresses an ink sheet with operation can be performed by a prior linear operaticm. 
heat to perform graded printmg on a receiving sheet of In this case, color reproduction is performed with high- 
papcr* precbkm color correction for colors reproducible by a 

After the above operation is performed for printing printer, and an optimal color reproduction is performed 
yellow on the ^ole receiving sheet of paper, the same with color predkrtion depending on the precision of the 
operation is performed for magenta, and cyan, so that a » operation for colors unreproduceable by a 

graded printing of a full color image is performed on the printer. 

reoei^ sheet of The present embodiments use a sublimation type 

thl^^^S^'.^S.'^l thermaltransfercolorprinter,butthemethodandappa. 
thc32x 32>02repri»mtativepomts^^ ratus for forming cok)r images in accordance with the 

"^°T^*^*^^ ''"*w *° S^^^K ink-den^ 55 trnveB^onarenotl^tothesub^^ 
signals, the mput signal (r, g, b) for each representative transfer color winter 

point is converted into a primary-colors signal (R,G,B) merma aunsi oo or inm • ,i,^„w 
bythefoUowing(20Xaidtt«Lk^lensi^^ . ^ this jontert. the term "mk^laisitjr shou^^ 

tiled from the rignal(R,G,B) by the «mie^^^ mterpreted not m ite iianow«^ 
asm the first emto^nent * 60 est meanmg. Namely, it mdicates "toner-density** m the 

case that color toners are used to reproduce color un- 
R=i" ages. 

Further, the interpolation operation for determining 
G«g^ an ix^-density signal in the fourth embodiment of the 

^ 65 present invention employs an 8-point interpolation 

®^ ^ method, using ratios of volumes, but the present inven- 

T £.^^1. 1 1 - - 1/n ^ n\ tion does not limit the method of the interpolation to a 

In CEurt, the pnmary-colors lummance signal (R,G,B) narticnilar ethod. 
obtained by (20) is converted into a first ink-density particular memoo. 
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' ''"Al&i^' 'the present mvoition has been folly de- 
scribed in connection with the preferred embodiments 
thereof with reference to the accompanying drawings, 
it is to be noted that various changes and modifications 
are apparent to those skilled in the art Such changes 
and modifications are to be understood as included 
within the scope of the present invention as defined by 
the appended dauns miless they depart therefr o m. 
What is claimed is: 

1. A method for forming color images, comprising; 
a first color correction process for performing a first 
color correction to an inpat color signal to obtain a 
target color chromatically equal to the input color 
and converting said target color into a first color 
density signal corresponding to said target color, 
a judgment process for judging whether or not the 
input color is reproducible by a printer having a 
color density, 
a second color correction process for performing a 
second color correction to said input color signal 
when judged in said judgment process that the 
input color is not reproducible by a printer, said 
second color correcdon converting said input 
color signal into a second color denaty signal 2s 
which is visually neatest to said target color within 
a color gamut reproducible by the printer, said 
second color correction process indoding a color 
rq>ioductio(tt prediction process for predicting 
colors reproducible by the printer with respect to 30 
said target color, a calculation process for calculat- 
ing evaluation values using at least one evaluation 
function for evaluating a chromatic rdation be- 
tween each of predicted colors and said target 
dolor, and a search process for seeking a best evalu- 
ation value among said evaluation values obtained 
in said calculation process to obtain said second 
color density signal based on the predicted color 
giving saod best evaluation value, and 
a color formation process for forming color images ^ 
while controlling the color dendty of the printer in 
accordance with said first color density signal 
when the input color is judged to be reproducible 
by the printer and said second color density signal 
when the mput color is judged to be unreproduci- 
ble by the printer. 
Z The method for forming color images according to 
claim 1 in which the input color is judged to be unre- 
producible by the printer if one of color components of 
said first color density signal has a density out of a range 
defined by a mininnnTn color density equal to that of a 
print sheet used for printing and a maximum color den- 
sity reproducible by the printer. 

3. The method for forming'color image according to 
claim 1 in which the color density signal to be con- 
verted is the first color density signal and said predic- 
tion process is performed by inverse operation of the 
first color density signal to a color signal in a color 
space of said input color signal. 

4. The method for forming color images according to 
claim 1 wherein a plurality of evaluation functions are 
provided and used selectively in accordaiu:e with color 
components of said input color signaL 

5. The method for forming color images according to ^5 
claim 4 wherein each of said plurality of evaluation 
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functions E(i)[i=l, . . . , n: n=the total number of a 
evaluation functions] is given by an equation depicted as 



wherein 



KOf c(f>: wctghtng coefTtdents 
AMI) = lo* - UO' 

So = («0*^ + 

5(0 = WO^ + Ki>»V 
AJ?„ - (SCO + 5a) • 1^ - e(i)\/2 

0(0 - txa-Hv(irMO*) 



Lo*f uo*, vo* coordinates of the target color m L*. u', 
V* space; 

L(i)*, u0*, V©* coordinates of the color to be repro- 
duced. 

6. The method for forming color images according to 
claim 1 wherein a plurality of neighborhood color den- 
sity signals are calculated from input color signals near 
to a considered input color signal and the color density 
signal correspondmg to the considered mput color sig- 
nal is corrected by a neighborhood calculation using 
said plurality of neighborhood color density signals 
whereby the color densty of the printer is controlled by 
the color density signal corrected by said neighbotlKXXl 
calculation. 

7. A color image forming apparatus for controlling a 
color printer which co mp rises a first correction means 
for performing a first color correction to an input color 
signal using upper bits of said input color signal to ob- 
tain a representative point of said input color signal and 
for converting said representative point of said input 
color signal into a first color density signal, 

a judgment means for judging whether or not said 
representative point of said input color signal is a 
color reproducible by a printer, 

a second color correction means for performing a 
second color correction to said representative 
point of said input color signal when judged by said 
judgment means that said representative point of 
said input color signal is not a color reproducible 
by the printer, said second color correction con- 
verting said representative point of said input color 
signal into a second color density signal which will 
reproduce a color visually nearest to said represen- 
tative point of said input color signal within a color 
gamut reproducible by the printer, 

a memory means for storing said first and second 
color density signals, 

a memory control means for addressing said memory 
means using upper bit information of said input 
color signals and reading the stored color density 
signal from said memory means, and 

an interpolation means for performing an interpola- 
tion operation to said stored color density signals 
read &om said memory means using lower bit in- 
formation of said input color information to 
obtain a final color density signal in response to 
which the color printer performs color printing 
operation. 

* • « * * 
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[57] ABSTRACT 

An input video image having an arbitrary resolution is 
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RESOLUTION DETERMINATION TABLE 



RESOLUTIONS 
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FREQUENCIES 


HORIZONTAL 
(kHz) 


VERTICAL 
(Hz) 


640x400 


24. 8 


56.4 


640X480 


37. 5 


72 


640x480 


31.5 


60 


800X600 


48 


72 


1 024x768 


56.5 


70 


1 600x1 200 
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1 

METHOD AND APPARATUS FOR SCALING 
UP AND DOWN A VmEO IMAGE 

BACKGROUND OF THE INVENTION 

1. Field of the iDvention 

The present inventioD relates to a method of and an 
apparatus for scaling up and down an input video image and 
displaying the resultant video image on a display device. 

2. Description of the Related Art 

In some cases, it is required to display video images 
generated by a computer on another dL^lay device, such as 
a liquid-crystal projector. In such a case, the computer 
should generate video signals according to the resolution of 
the di^lay device. In the description of this deification, 
the term "resohition" implies both a number of dots (that is, 
a number of pixels) in a horizontal direction of a video 
image and a number of lines (that is, a number of scanning 
lines) in a vertical direction. The number of dots in the 
horizontal direction is referred to as the horizontal 
resolution, whereas the nimiber of lines in the vertical 
direction is referred to as the vertical resolution. 

The resolution and the number of tones of a video image 
generated by a computer are restricted by the capacity of a 
video RAM (VRAM) in the computer. Hie number of tones 
is reduced for a display with a greater resolution (that is, a 
larger screen size), and increased for a display with a smaller 
resolution. When the display device has a significantly large 
screen size, it may be impossible to make the resolution of 
a video signal generated by the computer coincident with the 
resolution of the di^lay device. The similar problem arises 
when a video image generated by a device other than the 
computer (for example, a television image) is displayed on 
a display device other than a television receiver. 

SUMMARY OF THE INVENTION 

Accordingly, an object of the present invention is to 
convert any resolution of an input video image to a resolu- 
tion of a display device and display the video image with the 
converted resolution. 

The present invention is directed to a video image scaling 
apparatus for scaling up or down an input video image and 
displaying the scaled video image on a display device. The 
apparatus comprises: resolution determination means for 
analyzing an input video signal to determine a resolution of 
the input video signal; and scaling means for expanding or 
contracting a video image expressed by the input video 
signal so that the resolution of the video signal is made equal 
to a resohition of the display device. 

Since the resolution of a display device is known, a ratio 
of the resolution of tbe input video signal to the resolution 
of the display device can be obtained if the resolution of the 
input video signal is determined. Expansion or contraction 
of a video image by tbe ratio will make the resolution of a 
video signal equal to the resolution of the display device. 

In a preferred embodiment of tbe present invention, the 
resolution determination means comprises: resolution stor- 
age means for storing relations between the resolution of the 
input video signal and frequencies of synchronizing signals 
of the input video signal; frequency determination means for 
determining frequencies of the synchronizing signals of the 
input video signal; and means for reading out a resolution 
corresponding to the frequencies of tbe synchronizing sig- 
nals from tbe resolution storage means. 

When tbe relations between the resolutions of a video 
signal and frequencies of synchronizing signals are stored in 
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the resolution storage means, the resolution can be readily 
determined according to the frequencies of tbe synchroniz- 
ing signals. 

In accordance with an a^ct of the present invention, the 

5 apparatus further comprises: means for displaying a sign 
indicating that a resolution is unknown when the frequencies 
of the synchronizing signals of the input video image are not 
stored in the resolution storage means; and resolution setting 
means for setting a value of the unknown resolution of the 

10 input video signal and registering a relation between the 
resolution and the frequencies of the synchronizing signals 
of the input video signal in tbe resolution storage means. 

This aspect allows to convert the resolution of an input 
image signal even if the input video signal has synchroniz- 
ing signals of unknown frequencies. 

The scaling means comprises: a first buffer memory for 
temporarily storing the input video signal; a frame memory 
in which a video signal read out of the fiist buffer memory 
is written; a second buffer memory for temporarily storing a 
video signal read out of the frame memory; and memory 
control means for giving a write address to the frame 
memory while successively reading out video signals from 
the fiist buffer memory to write the video signal read out of 

^ the first buffer memory into the frame memory, and for 
giving a read address to the fiame memory to read out the 
video signal from the frame memory and transfer the video 
signal to the second buffer memory, and wherein tbe 
memory control means comprises: means for expanding or 

^ contracting a video image read out of the frame memory by 
adjusting the read address given to the frame memory. 

The present invention is also directed to a method of 
scaling up or down an input video image and displaying tbe 
scaled video image on a di^lay device. The method com- 

35 prises the steps of: (a) analyzing an input video signal to 
determine a resolution of the input video signal; and (b) 
expanding or contracting a video image expressed by the 
input video signal so that the resolution of the video signal 
is made equal to a resolution of the display device. 

40 These and other objects, features, aspects, and advantages 
of tbe present invention will become more apparent from the 
following detailed description of tbe preferred embodiments 
with the accompanying drawings. 

45 BRIEF DESCRIPTION OF THE DRAWINGS 

FIG, 1 is a block diagram illustrating the structure of a 
liquid-crystal projector as a first embodiment according to 
the present invention; 

FIG. 2 schematically illustrates ftindions of a video scaler 
36; 

FIG. 3 is a block diagram illustrating the internal structure 
of the video scaler 36; 

FIG. 4 shows the contents of a resolution determination 
55 table; 

RG. 5 shows a wave form of a composite video signal; 

RG. 6 is a block diagram ilhistrating the internal structure 
of a scaling unit 70; 

RG. 7 is a timing chart showing a process of generating 
vertical addresses; 

RGS. SAand 8B show a concrete procedure of expanding 
a video image; 

RG. 9 is a timing chart showing a process of generating 
55 horizontal addresses; 

RG. 10 is a block diagram illustrating the internal struc- 
ture of a latch error elimination circuit 150; 
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FIG. 11 is a block diagram illustrating the structure of a resolution in the liquid-crystal projector is independent of 

down converter as a second embodiment according to the the resolution of the input video signal VPC. Accordingly, a 

present invention; and video image originally generated by the personal computer 

nCS. 12A, 12B1, 12B2, and L2B3 show a process of can be reproduced on the LCD panel 40 to cover its 

expanding and contracting a video image by multiplying 5 whole display area regardless of the r^ludon and he 

.J . jf J «r- . ir ^ ^ number of tones of the unage which arc detcrnuned by the 

read addresses by a predetermined coeffiaent K. ^^^^^ computer 100. 

DESCRIPTION OF THE PREFERRED ^ a block diagram illustrating the internal structure 

EMBODIMENT video scaler 36. The video scaler 36 includes a first 

jQ color conversion unit 50, a write synchroni2dng signal gen- 
FIG. 1 is a block diagram illustrating the structure of a crator 52, an input FIFO buffer 54, a DRAM controller 56, 
liquid-crystal projector as a first embodiment according to an address controller 58, a CPU access controller 60, two 
the present invention. The liquid-crystal projector projects output FIFO buffers 61 and 62, a filter unit 64, a second color 
video images generated by a personal computer 100 on a conversion unit 66, and a read synchronizing signal genera- 
large-size screen (not shown). The liquid-crystal projector tor 68. As shown in FIG. 3, the frame memory 34 is 
includes a CPU 20, a main memory 22, an input panel 24 constructed as a dynamic RAM in this embodiment. The 
functioning as input means, an A-D converter 32, a frame DRAM controller 56 is a circuit for controlling a process of 
memory 34, a video scaler 36, LCD drivers 38, LCD panels writing video signals into the firame memory 34 and a 
(liquid-crystal panels) 40, and a light source 42. The fi^ame process of reading out video signals from the frame memory 
memory 34 includes three memory planes for storing R, G, ^* 

and B signals. respecUvely. The LCD drivers 38 and the ^ The digital video signal DPC output from the A-D con- 
LCD panels are also provided for the R, G, and B signals. ^erter 32 shown in FIG. 1 is given to the first color 
„, ^„ -^n t r ^ . * .* * conversion unit 50, which cames out a color conversion to 
The CPU 20 funcuons as a frequency daermmaUon urn ^ necessary. By way of example, when the 
26 for determmmg a frequency of a syn^onmng signal ^ DPC is an YCrCb signal, the first 
SYNC given by the personal computer 100 and as a reso- ^ ^^^^ conversion unit 50 converts the YCrCb signal to an 
lution determination umt 28 for determuung a resolution RGB signal 

corresponding to the frequency of the synchronizing ^gnal synchronizing signal SYNC generated by the per- 

SYNC. The CPU 20 executes computer program codes g^^,^ computer 100 includes a horizontal synchronizing 

stored in the main memory 22 to implement these functions. signal HSYNCl and a vertical synchronizing signal 

The A-D converter 32 converts an analog video ^gnal ^ VSYNCl. The write synchronizing signal generator 52 has 

VPC generated by the personal computer 100 to a digital an internal PLL circuit (not shown), which multiplies the 

video signal DPC and transmits the digital video signal DPC frequency of either the horizontal synchronizing signal 

to the video scaler 36. The video scaler 36 receives the HSYNCl or the vertical synchronizing signal VSYNCl by 

digital video signal DPC as well as the synchronizing signal Nq to generate a dot dock signal DCKl. The dot clock signal 

SYNC ou^jut from the personal computer 100. In the 35 DCKl indicates an update timing of a dot position in the 

description of this specification, the term "video signals" horizontal direction. The dot clock signal DCKl as weU as 

may represent video signals in a narrow sense that do not horizontal synchronizing signal HSYNCl and the ver- 

inchide synchronizing signals, and also those in a broad ^cal synchronizing signal VSYNCl are supplied to the 

sense that inchide synchronizing signals. address controller 58. ^ ^ 

T, .J I ^ * *u • * J- % I -1 1 The video signal converted by the first color conversion 

jme video scaler 36 wntest^^ unit 50 is tem^rarily stored in the HFO buffer 54 and 

DPC mto the frame memory 34 while reading out a video ^^^^ the frame memory 34 by the DRAM controller 

signal from the frame memory 34 and supplymg the video pjpQ ^^^^^ 54 ^j^j ^^e timing of the 

signal to the LCD driver 38. In the course of wnting or writing operation. The writing operation into the frame 

reading procedure, the video scaler 36 expands or contracts memory 34 is carried out synchronously with the write 

a video image, so as to make the resolution of the video 45 synchronizing signals (DCKl, HSYNCl, and VSYNCl) 

signal coincident with a standard resolution of the LCD output from the write synchronizing signal generator 52. 

panel 40. The LCD driver 38 reproduces a video images that Each dot position (or horizontal address) is updated syn- 

is transmitted from the video scaler 36 on the LCD panel 40. chronously with the dot clock signal DCKl, while each 

The video images reproduced on the LCD panels 40 are scanning line position (vertical address) is updated synchro- 

finally projected as a color image on the screen by means of 50 nously with the horizontal synchronizing signal HSYNCl. 

an optical system including the light source 42. Each frame or each field is updated synchronously with the 

FIG, 2 schematically illustrates the functions of the video vertical synchronizing signal VSYNCl. The DRAM con- 

scaler 36. As shown in the left half of FIG. 2, video images troller 56 also reads out video signals stored in the frame 

generated by the personal computer 100 may have a variety memory 34 and writes the input video signals alternately 

of resohitions (for example, 640 dots by 400 lines, 640 dots 55 into the two FIFO buffers 61 and 62. The reading-out 

by 480 lines, 800 doLs by 600 lines, 1,024 dots by 768 lines, operation from the frame memory 34 is carried out synchro- 

and 1,600 dots by 1,200 lines). The standard resolution of nously with read synchronizing signals (DCK2, HSYNC2, 

the LCD panel 40 Ls, on the other hand, fixed to a prede- and VSYNC2) generated by the read synchronizing signal 

termined value. In the example of FIG. 2, the standard generator 68. llie read synchronizing signals (DCK2, 

resolution is 800 dots by 600 lines. The video scaler 36 60 HSYNC2, and VSYNC2) are also supplied to the LCD 

accordingly expands or contracts the input video signal VPC driver 38 to be used as display synchronizing signals for the 

in order to generate a video signal having the standard LCD panel 40. The address controller 58 is a circuit for 

resolution of the LCD panel 40. When the video signal VPC generating a write address and a read address and supplying 

generated by the personal computer 100 is input into the the write and read addresses to the DRAM controller 56. The 

liquid-crystal projector of this embodiment, a video image 65 address controller 58 further includes a scaling unit 70 for 

expressed by the video signal VPC will be reproduced on the expanding or contracting (or scaling up or down) a video 

whole screen of the LCD panels 40. This means that the image. 
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One line of video signals read out from ibe frame memory 
34 are written alternately into the two output FIFO buffers 
61 and 62. In the mean time, video signals are read out from 
the buffer which is not under the writing operation, to be 
supplied to the filter unit 64. The filter unit 64 is a circuit for 5 
carrying out a variety of filtering processes, such as y 
correction (conversion of input/output tones) and left-to- 
right and top-to-bottom inversions of video images. The 
filtered video signal undergoes the color conversion in the 
second color conversion unit 66, if necessary, to be con- 
verted to an output video signal DOUT. llie output video 
signal DOUT is then supphed to the LCD driver 38 (see FIG. 
1). 

The CPU 20 shown in FIG. 1 has access to the re^)ective 
elements in the video scaler 36 via the CPU access controller 15 
60 shown in FIG. 3. In measuring the frequency of the 
synchronizing signal SYNC corresponding to the input 
video signal VPC, the CPU 20 receives the signals output 
&om the write synchronizing signal generator 52 via the 
CPU access controller 60. The CPU 20 first functions as the 20 
frequency determination unit 26 (see FIG. 1) to measure the 
&equeDcies of the horizontal synchronizing signal HSYNCl 
and the vertical synchronizing signal VSYNCl, which are 
supplied to the write synchronizing signal generator 52. The 
CPU 20 then functions as the resolution determination unit 25 
28 to determine the resolution of the input video image VPC 
based on the measured frequencies. 

FIG. 4 shows a resolution determination table indicating 
relations between the resolutions and the frequencies of the 
synchronizing signals. The relations between the various 30 
resolutions (the number of dots by the number of lines) and 
the frequencies of the horizontal synchronizing signal and 
the vertical synchronizing signal are registered in the reso- 
lution determination table, which is stored in the main 
memory 22. The frequency of an operation clock of the CPU 35 
20 is at least tens of MHz while the frequency of the 
horizontal synchronizing signal is tens of kHz, and the 
frequency of the vertical synchroniziag signal several is tens 
of Hz. The CPU 20 can thus execute the computer program 
codes to implement the function of the frequency determi- 40 
nation unit 26 to measure these fipequeocies with a suffi- 
ciently high accuracy. In accordance with a concrete 
procedure, the CPU 20 carries out the oounting-up operation 
at a regular interval and obtains a count between edges (such 
as Calling edges) of the horizontal synchronizing sigxial 45 
HSYNCl. The CPU 20 then calculates the frequency of the 
horizontal synchronizing signal HSYNCl from the count. 
The frequency of the vertical synchronizing signal VSYNCl 
can be determined in a similar manner. After determining the 
frequencies of the synchronizing signals HSYNCl and 50 
VSYNCl, the resolution determination unit 28 determines 
the corresponding resolution by referring to the resolution 
determination table (FIG. 4). 

As shown in FIG. 4, plural combinations of the frequen- 
cies of synchronizing signals may correspond to an identical 55 
resolution. It is accordingly desirable to register relations 
between the resolutions and the frequencies used in a 
number of commercially available apparatuses as many as 
possible into the resolution determination table. There is, 
however, still a possibility of receiving a video signal having 60 
a frequency not registered in the resolution determination 
table. In such a case, the CPU 20 shown in FIG. 1 may make 
a display on the LCD panel 40 (or on a display unit of the 
input panel 24), showing that the frequency of the input 
video signal VPC has not yet been registered. The user then 65 
sets the resolution (the number of dots by the number of 
lines) of the input video signal VPC with the input panel 24, 
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thereby registering the relation between the frequency and 
the resolution into the resolution determination table. In 
order to realize this process, it is desirable to store the 
resolution determination table in a write-enable memory, 
such as a RAM or a flash memory. 

The resolution of the input video signal VPC may be 
determined on the basis of not only the frequencies of the 
horizontal synchronizing signal and the vertical synchroniz- 
ing signal but on period widths H^ and H,^of the horizontal 
and vertical synchronizing signals and on the kind of inter- 
lacing. FIG. 5 shows the period widths H,, and Hy of the 
horizontal synchronizing signal and the vertical synchroniz- 
ing signal. For convenience of illustration, FIG. 5 shows a 
wave form of a composite video signal. Determination of the 
resolution of the input video signal based on the frequencies 
of the synchronizing signals as well as their period widths 
H|,and H^and the state of interlacing can effectively reduce 
the possible errors that may be made in the determination. 

The horizontal resolution and the vertical resolution deter- 
mined by the resolution determination unit 28 are given to 
the address controller 58 via the CPU access controller 60 
(see FIG, 3). The scaling unit 70 in the address controller 58 
carries out expansion or contraction of a video image as 
described before along with FIG. 2, in order to convert the 
horizontal and vertical resolutions to the standard resolu- 
tions of the LCD panel 40. 

FIG. 6 is a block diagram illustrating the internal structure 
of the scaling unit 70. The scaling unit 70 includes a PLL 
circuit 142, a frequency divider 144, a horizontal address 
generator 146, a vertical address generator 148, a 3-state 
buffer 160, and an inverter 162. A data latch 164 shown in 
FIG. 6 is a circuit included in the DRAM controller 56. The 
horizontal address generator 146 includes a latch error 
elimination circuit 150, a first counter 152, and a first latch 
154. The vertical address generator 148 includes a second 
counter 156 and a second latch 158. 

The PLL circuit 142 receives the horizontal synchronizing 
signal HSYNC2, which is generated for the reading-out 
operation, aixi generates a second dot clock signal DCKX 
having the frequency of N times the frequency of HSYNC2. 
The frequency divider 144 receives the dot clock signal 
DCK2, which is also generated for the reading operation, 
and divides the frequency of DCK2 by M to generate a line 
increment signal LSHOC The preset values N and M in the 
PLL circuit and the frequency divider 144 are used to 
convert the resolution of the input video signal VPC to the 
resolution of the LCD panel 40, and are reflectively deter- 
mined by the CPU 20. A concrete process of determining the 
preset values N and M will be described later. 

FIG. 7 is a timing chart showing operation of the vertical 
address generator 148. After being reset by the vertical 
synchronizing signal VSYNC2 for the reading operation 
(FIG. 7(a)), the second counter 156 counts the number of 
pulses in the line increment signal UNCX. A count HC on 
the second counter 156 (FIG. 7(d)) is latched at a rising edge 
of the horizontal synchronizing signal HSYNC2 and given 
as a vertical address VADD to the 3-state buffer 160. In the 
example of RG. 7{e\ the vertical address VADD is updated 
as 0, I, 1, 2, . . . 

FIGS. 8A and 8B show a concrete process of expanding 
a video image. FIG. SAshows video data stored in the frame 
memory 34, and FIG. 8B shows expaixled video data. 
Numerals written in the tables represent the values of video 
data. In the timing chart of FIG. 7(e), video data are read out 
from the frame memory 34 such that: a video image on a 
scarming line of VADD-0 is read out once, a video image on 
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a scaoniDg line of VADO-1 twice, a video image od a 
scanning line of VADD-2 twice, and the like. The video 
image thus read out is accordingly expanded in the vertical 
direction as shown in RG. 8B, A vertical magnification 
MV2 is given as a ratio of a frequency fHSYNC2 of the 5 
horizontal synchronizing signal HSYNC2 to a frequency 
fUNCX of the line increment signal UNCX. The video 
image can be expanded by an arbitrary magnification in the 
vertical direction by adjusting the preset value M in the 
frequency divider 144 (FIG. 6). The video image will be 
contracted in the vertical direction when the value of the 
magnification MV2 is less than 1. 

FIG. 9 is a timing chart showing an operation of the 
horizontal address generator 146. The latch error elimination 
circuit 150 (FIG. 6) generates a third dot clock signal 
DCKXX (FIG. 9(e)) from the first and the second dot clock 
signals DCK2 and DCKX (FIGS. 9(b) and 9(d)). 

FIG. 10 is a block diagram illustrating the internal struc- 
ture of the latch error elimination circuit 150. The latch error 
elimination circuit 150 includes a delay circuit 170, an 
exclusive NOR (EXNOR) circuit 172, and a D-type flip-flop 20 
174. An output signal DKFF of the EXNOR circuit 172 is an 
inversion of an exclusive OR of the first dot clock signal 
DCK2 and a signal obtained by delaying the dot clock signal 
DCK2 by a predetermined time period. The signal DKFF 
thus represents timings of rises and falls of the first dot ckx;k 25 
signal DCK2 as shown in FIG. 9(c). 

The output signal DKFF of the EXNOR circuit 172 is 
supplied to a clock input terminal of the flip-flop 174, whDe 
the second dot clock signal DCKX is given to a D-input 
terminal of the flip-flop 174. The third dot clock signal 30 
DCKXX output from the flip-flop 174 thus represents the 
level of the second dot clock signal DCKX at a rising edge 
of the output signal DKFF of the EXNOR circuit 172 as 
shown in FIG. 9(e). The third dot clock signal DCKXX has 
the frequency identical with that of the second dot clock 35 
signal DCKX. The output signal DKFF of the EXNOR 
circuit 172 rises after a predetermined delay time from an 
edge of the first dot clock signal DCK2, and the timing of the 
level diange of the third dot clock signal DCKXX is delayed 
by the predetermined delay time from the edge of the first 40 
dot clock signal DCK2 accordingly. The latch error elimi- 
nation drcuit 150 generates the third dot clock signal 
DCKXX, in order to prevent the value of a horizontal 
address latched by the fiiist latch 154 from being unstable as 
discussed later in detail. 45 

After being reset by the pulse of the horizontal synchro- 
nizing signal HSYNC2, the first counter 152 of the hori- 
zontal address generator 146 (FIG. 6) counts up the number 
of pulses of the third dot clock signal DCKXX generated by 
the latch error elimination circuit 150 and supplies a count 50 
DC (FIG. 9(f)) to the first latch 154. Since the third dot clock 
signal DCKXX and the second dot clock signal DCKX have 
identical firequencies as mentioned above, the count DC of 
the first counter 152 practically indicates the number of 
pulses of the second dot clock signal DCKX. The first latch 55 
154 latches the count DC synchronously with the first dot 
clock signal DCK2, and gives the latched count as a hori- 
zontal address HADD (FIG. 9(g)) to the 3-state buff^cr 160. 
The horizontal address HADD accordingly represents the 
number of pulses of the second dot clock signal DCKX and 60 
is updated at every rising edge of the first dot clock signal 
DCK2. The value of the horizontal address HADD can be 
updated in a predetermined manner by adjusting a frequency 
fDCK2 of the first dot clock signal DCK2 and a frequency 
fDCKX of the second dot clock signal DCKX. In the 65 
example of FIG. 9(g), the value of the horizontal address 
HADD is varied as 0,0,1, . . . 
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The tables of FIGS. 8A aiKl 8B discussed above show a 
process of expanding a video image according to the hori- 
zontal address HADD in FIG. 9(g). Ilie timing chart shown 
in FIG. 9 corresponds to timings of generating addresses in 
the horizontal direction on an upper-most scanning line 
having the vertical address of VADD^O. The horizontal 
address HADD is updated as 0,0,1 ... as shown in FIG. 9(g). 
Video data of the respective pixels existing on this scanning 
line are successively read out from the frame memory 34 o 
that the video data of the pixel having the horizontal address 
HADD»0 is read out twice, the video data of the pixel 
having the horizontal address HADD»1 is read out once, and 
the like. 

As discussed above, the horizontal address HADD 
depends upon the relation between the frequencies of the 
two dot clock signals DCK2 and DCKX. A video image can 
thus be expanded or contracted in the horizontal direction by 
adjusting the fr^uencies of these dot clock signals DCK2 
and DCKX. A magnification MH2 of a video image in the 
horizontal direction is given as the ratio of the frequency 
fDCK2 of the first dot clock signal DCK2 to the frequency 
fDCKX of the second dot clock signal DCKX as shown in 
the bottom of FIG. 8. A video image can accordingly be 
expanded or contracted by an arbitrary magnification in the 
horizontal direction by adjusting the preset value N in the 
PLL circuit 142. 

The reason why the latch error elimination circuit 150 is 
used to generate the signal DCKXX is as follows. As shown 
in FIG. 9(f), the count DC on the first counter 152 is varied 
synchronously with each rising edge of the third dot clode 
signal DCKXX (FIG. 9(e)) after the horizontal synchroniz- 
ing signal HSYNC2 (FIG. 9(a)) is returned to the high level. 
As discussed previously, an edge of the third dot clock signal 
DCKXX is delayed by a predetermined time period from an 
edge of the first dot clock signal DCK2. The latch timing in 
the first latch 154 thus does not overlap the timing of 
variation in count DC, so that the value of the horizontal 
address HADD is made stable. 

As discussed above, the magnification MH2 in the hori- 
zontal direction and the magnification MV2 in the vertical 
direction can be set independently as shown in the bottom of 
FIG. 8, by adjusting the preset value N of the PLL circuit 
142 and the preset value M of the frequency divider 144 
shown in FIG. 6. A video image can be reproduced on the 
whole screen of the LCD panel 40 by setting the magnifi- 
cation MH2 in the horizontal direction equal to the ratio of 
[the horizontal resolution of the LCD panel 40] to [the 
horizontal resolution of the input video signal VPC] and by 
setting the magnification M V2 in the vertical direction equal 
to the ratio of [the vertical resolution of the LCD panel 40] 
to [the vertical resolution of the input video signal VPC]. 

FIG. U is a block diagram illustrating the structure of a 
down-converter as a second embodiment according to the 
present invention. The down-converter has a video signal 
selection unit 200 in addition to the input unit of the 
liquid-crystal projector shown in FIG. I. The down- 
converter further includes a video encoder 202 in place of 
the LCD driver 38, and a variety of output devices (such as 
a television receiver 204, a video player 206, and a 
CD-RAM 208) in place of the LCD panel 40 and the light 
source 42. 

The video signal selection unit 200 receives two televi- 
sion signals STVl and STV2 as well as video signals (VPC, 
SYNC) generated by a personal computer, and selects one of 
the received signals. The television signals STVl and STV2 
are composite video signals including synchronizing signals. 
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When the video sigaai selection unit 200 selects a composite FIG. 12B1 shows an example of the displayed image 

video signal, a decoder (not shown) in the video signal when the coefficients Kx and Ky are greater than 1.0 (for 

selection unit 200 generates component video signals VIN example, Kx«-Ky«2,0). When the original horizontal address 

and a synchronizing signal SYNC from the selected com- XADD is increased one by one, such as 0,1^, . . . , the 

posite video signal. 5 converted horizontal address XADD is varied as 0,2,4, , . . 

The video encoder 202 generates a composite video according to Equation (2a) given above. The vertical address 
signals from a digital video signal DOUT and the reading- YADD is converted in the same manner. Video data arc read 
out synchronizing signals (DCK2, HSYNC2, and VSYNC2) out fiom the frame memory 34 according to the converted 
output from the video scaler 36. The composite video signal read addresses XADD and YADD, so that a contracted video 
thus generated is supplied to the television receiver 204 and to image is displayed as shown in FIG. 12B1. The horizontal 
the video player 206. In order to write a video image into the magnification and the vertical magnification in this contract- 
CD-RAM 208 (write-enable compact disk unit), the video ing process are respectively equal to 1/Kx and 1/Ky. 
encoder 202 does not generate a composite video signal but When the coefficients Kx and Ky are equal to 1.0, a video 
directly supplies the digital video signal DOUT and the image in the frame memory 34 is displayed without any 
reading-out synchronizing signals to the CD-RAM 208. The 15 expansion or contraction as shown in FIG. 12B2. 
video scaler 36 can change the resolution of a video image P^G- 12B3 shows an example of the displayed image 
to a desired resolution as discussed previously. When the when the coefficients Kx and Ky are smaller than 1.0 (for 
user ^ifies a desired resolution, the video scaler 36 can example, Kx-Ky-0.7). When the original horizontal address 
output video images with the desired resolution correspond- XADD is increased one by one as 0,1,23, . . . , the converted 
ing to the various output devices. The apparatus of FIG. 11 20 horizontal address XADD is varied as 0,0,1,2, . . . The 
is called downnconverter because it can convert a variety of vertical address YADD is converted in the same manner, 
input video signals down to a variety of output video signals. Video data are read out from the frame memory 34 accord- 
Hie present invention is not restricted to the above "^g ^he converted read addresses XADD and YADD^^so 
embodiments or applications. There may be many that an expanded video image is displayed as shown m FIG. 
modifications, changes, and alterations without departing ^ 12B3. j 
from the scope and spirit of the main characteristics of the Possible to set arbitrary values to Kx and Ky 
inventions foUows. mdepeodenUy. 

/iv-TL c r»u <^ J » - * (4) When a high-speed read/wnte memory, such as a 

(1) The functions of the frequency determination unit 26 ^ ' _ _ fTi • ^ r .u *^ ia 
i . ^. . , .J* •* ->o / 171/- i\ 1 synchronous DRAM, is used for the frame memory 34, 

and the resohition determmabon umt 28 (see FIG. 1) real- . < . . ' . . ^ . . . ' . 

. , , . J - *L 1. J* 30 hiRh-speed reading and wntmg of video signals can be 

ized by the computer program codes m the above embodi- ^ & o 

^ , 1- J L ^ i_ • •* earned ouL 

ments may be realized by hardware circuits. l .i. . • *- u u ^u^a 

^ ^ Although the present mvention has been descnbed and 

(2) In the above embodiments, image expansion and Ohistrated in detail, it is clearly understood that the same is 
contraction are earned out when video images are read out illustration and example only and is not to be 
from the frame memory 34. The image expansion and 35 lakenby way of Umitation, the spirit and scope of the present 
contraction may, however, be executed when video images invention being limited only by the terms of the appended 
arc wntten into the fr:ame memory 34. claims 

(3) Any technique other than the frequency control dis- What is claimed is: 

cussed above may be applied to the image expanaon and i ^ video image scaling apparatus for receiving a video 

contraction. For example, they can be attained by multiply- 40 image output as a video signal in a first format for a first 

ing the read address or the write address by a predetermined display device and outputting the video image in a second 

coefficient to change the addresses so that the image is format for a second display device, said apparatus compris- 

expanded or contracted according to the changed addresses. {Qg. 

nCS. 12A and 12B1-12B3 show a process of expaiiding bcqoc^ determination means for analyzing the video 

and contractmg a video unage by multiplying read addrc?* « ^ determine a frequency of synchronizing sig- 

by a predetermined cocffiaent K. FIG. 12A sh^ a vKleo ^ ^ ^ ^^^j. 

image stored m the frame memory 34, whereas FIGS. 12B1 . ^ , . 

thrcSgh 12B3 show expanded or contracted video images. In size deteraiination means for determming an miage 

the drawing, Di,j represents video data written at an address ^« °^ ^"SP^iby analyzing the frequency of 

(y) in the frame memory 34. jo the synchronizing signals of the video signal output m 

In the example of RGS. 12A and 12B1-12B3, it is ^^""""^ , . 

assumed that a memory resoluUon is defined by Mx (dots) «=almg means for scalmg the video miage expressed by 

by My (lines) and a display resolution Nx (dots) by Ny y"**^ ^"Sf"^ first format by utUmng the 

(lines). The coefficients K(Kx,Ky) by which the addresses synchronizmg signab and the determmed image size so 

are multiplied are given as foUows: 55 video signal is output in the second format of 

said second display device. 

KxmMxiNx (la) 2. A video image scaling apparatus in accordance with 

KymMyiNy (lb) claim 1, wherein said image si/e determination means 

comprises: 

A read address (XADD,YADD) used for reading out ^ storage means for storing relationships which 

video data from the frame memory 34 is converted to a new identify the image size of the video signal in the first 

read address (XADD,YADD) by the foUowing equations: ^^^j ^j^sed on the frequency of the synchronizing 

XADih^NTQCjDMDD) (2a) signals of the video signal; and 

, ^. , means for determining the image size of the video signal 

^ 65 m the first format by referencing said image size 

wherein the operator INT( ) represents an operation of storage means according to the frequency of the syn- 

taking an integ^ portion of the value in parentheses. chronizing signals of the video signal. 
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3. A video image scaling apparatus in accordance with 
claim 2y said apparatus further comprising: 

means for displaying a sign indicating that an image size 
is unknown when the frequencies of the synchronizing 
signals of the video image are not stored in said image 5 
size storage means; and 

image size setting means for setting a value of the 
unknown image size of the video signal and registering 
a relation between the image size and the frequencies of 
the synchronizing signals of the video signal in said lo 
image size storage means. 

4. A video image scaling apparatus in accordance with 
claim 1, wherein said scaling means comprises: 

a fiist buffer memory for temporarily storing the input 
video signal; is 

a frame memory in which a video signal read out of said 
first buffer memory is written; 

a second bu£fer memory for temporarily storing a video 
signal read out of said frame memory; and 

memory control means for giving a write address lo said ^ 
frame memory while successively reading out video 
signals from said first buffer memory to write the video 
signal read out of said first buffer memory into said 
frame memory, and for giving a read address to said 
frame memory to read out the video signal from said 25 
frame memory and transfer the video signal to said 
second buffer memory, and wherein 
said memory control means comprises: 

means for expanding or contracting a video image 
read out of said frame memory by adjusting the 30 
read address given to said frame memory. 

5. A method for receiving a video image output as a video 
signal in a first format for a first display device and output- 
ting the video image in a second format for a second display 
device, said method comprising the steps of: 35 

(a) analyzing the video signal to determine a frequency of 
synchronizing signals when the video signal is output 
in the first format; 

(b) determining an image size of the video signal by 
analyzing the frequency of the synchronizing signals of 40 
the video signal output in the first format; and 

(c) scaling the video image expressed by the video signal 
in the first format by utilizing the synchronizing signals 
aiKi the determined image size so that the video signal 

is output in the second format of said second display 45 
device. 

6. A method in accordance with claim 5, wherein said step 
(b) comprises the steps of: 

storing, in a memory, relationships which identify the 
image size of the video signal in the first format based 5Q 
on the frequency of the synchroaizing signals of the 
video signal; and 

determining the image size of the video signal *n the first 
format by referencing said memory according to the 
frequency of the synchronizing signals of the video 
signal. 

7. A method in accordance with claim 6, further compris- 
ing the steps of: 

di^laying a sign indicating that an image size is unknown 
when the frequencies of the synchronizing signals of ^ 
the video image are not stored in said memory; and 

setting a value of the unknown image size of the video 
signal and registering a relation between the image size 
and the frequencies of the synchronizing signals of the 
video signal in said memory. 

8. A method in accordance with claim 5, wherein said step 
(b) comprises the steps of: 
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writing the input video signal into said frame memory; 
and 

giving a read address to said frame memory to read out the 
video signal from said frame memory while adjusting 
the read address to expand or contract a video image 
read out of said frame memory. 

9. A video image scaling apparatus for receiving a video 
image output as a video signal in a first format for a first 
display device and outputting the video image in a second 
format for a second display device, said apparatus compris- 
ing: 

a video signal input for receiving a video signal, in the 
first format, including synchronizing signals; 

a synchronizing signal frequency analyzer receiving the 
video signal from the video signal input and determin- 
ing a frequency of the synchronizing signals when the 
video signal is output in the first format; 

an image size determination unit for determining an 
image size of the video signal by analyzing the fre- 
quency of the synchronizing signals being applied to 
the video signal input; and 

a scaling unit for scaling the video image expressed by the 
video signal in the first format by utilizing the synchro- 
nizing signals and the determined image size so that the 
video signal is output in the second format of said 
second display device. 

10. A video image scaling apparatus in accordance with 
claim 9, wherein the image size determination unit com- 
prises: 

a memory unit for storing relationships which identify the 
image size of the video signal in the first format based 
on the frequency of the synchronizing signals of the 
video signal; and 

a lookup unit for looking up frequencies in the memory 
unit to determine the image size of the video signal 
output in the first format. 

11. A video image scaling apparatus in accordance with 
claim 10, further comprising: 

an iiulicator for indicating that an image size is unknown 
when the frequency of the synchronizing signals are not 
stored in the memory unit; and 

a memory unit updating unit for setting in the memory 
unit (1) a value of the unknown image size of the video 
signal and (2) a relation between the unknown image 
size and the frequency of the synchronizing signals. 

12. A video image scaling apparatus in accordance with 
claim 9, wherein said scaling unit comprises: 

a first buffer memory for temporarily storing the video 
signal; 

a frame memory in which a video signal read out of said 
first buffer memory is written; 

a second buffer memory for temporarily storing a video 
signal read out of said frame memory; and 

a memory controller for applying a write address to said 
frame memory while successively reading out video 
signals from said first buffer memory to write the video 
signal read out of said first buffer memory into said 
frame memory, and for applying a read address to said 
frame memory to read out the video signal from said 
frame memory and transfer the video signal to said 
second buffer memory, and wherein 

said memory controller includes a read address updating 
unit which expands or contracts a video image read out 
of said frame memory by adjiisting a read address given 
to said frame memory. 

* « * ♦ * 
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VIDEO SAMPLING STRUCTURE 4:4:4. All "even" samples along the register line 

CONVERSION IN BMME (counting the first sample as sanaple 0) already have an 

accurate U,V co-sited sample which can remain 

CROSS-REFERENCE TO RELATED unchanged. For "odd" samples a new vahie must be 

APPLICATIONS 5 calculated. The vahic can be calculated by interpolating 

The present application may relate to co-pending appli- horizontally between the neighboring two "even" 

cation Ser. No. 09/960^72, filed Sep. 21. 2001 and Ser. No. sample values for U and V; or 

09/878^94, filed Jun. 11, 2001, which are hereby incorpo- (iii) interleaving of daU is required to combine the 

rated by reference in their entirety. separate Y and C daU blocks into 24-bpp YUV pixels 

10 for conversion firom 4:4:4 to graphics YUV. Cbnver^on 
from 4:4:4 to graphics RGB is similar to conversion to 

The present invention relates to a method and/or archi- graphics YUV, and followed by a color conversion step 

tecture for integrating video and graphics processing in a to change from YUV to RGB. 

single processor generally and, more particularly, to data It is therefore generally desirable to provide a method 

modification used to allow changes in sampling structvire for and/or architecture capable of video/graphics sampling 

video to graphics data conversion. structure conversion. 

BACKGROUND OF THE INVENTION SUMMARY OF THE INVENTION 

Digital video and/or graphics systems sampte and store ^^^^ ^ ^ comprising 

videomoneofanumberofformals,eachofwhichmclude 20 ^ ^aia rLlification circuit and a composite circuit TTie data 

a Y and a C component (the Y component refers to a .^^^^tx circuit may be configured to generate a first and 

himinance component of the video and the C component . . , _ ' . T * * -a a « 

- . . , , • « second video component m response to a video data stream. 

refers to a chrommance component (i.c_ a U, V pair)). For „, f . j . . 

-.1- -J r ^ V J / - 11 The composite circmt may be configured to present an 

convemence with video formats, the Y data is normally . * * u • . i • *u a .u- 

. , , . . ^ . . ^ . oc output graphics stream by mterleavmg the first and the 

stored as a block m one region of memory and the C data ^ , ,j ^ 

/*«- f T ^/ • \ • * J . i_ 1- • I- *. second video component 
(the U, V pairs) is stored as a sq)arate block. For simplicity, 

the present application wiU refer to all video and gr^hics o*^^ advantages of the present mven- 

components/samples as 8-bit quantities. providing a method and/or architecture for 

rteferring to HG. 1, a conventional video format 4:2:0 is * sampling strucdire conversion engme 

. JT^ -jf .^mu r> t f 30 that may also permit mterleavmg of Y and C component 

shown. The video format 4:2:0 has one C sample (i.e„ one ^ ., ."^ - ^ . - , -j jf . a.* u *u 

„ , . . - f r V 1 video data mto a smgle video data stream which may then 

U, V sample pair) for every group of fiour Y samples. The . ^ ^ * l_* j * r . 

i -.f»u>» . c be easdy converted to a graphics data format 

effective sample pomt for the C component is m a center of ^ ^ 

a scpiarc of Y components. In the 4:2:0 format of HG. 1, an DESCRIFHON OF THE DRAWINGS 
amount of Y compoisenl storage required (Le., 4) is twice 

that required for the C component (Le., 2), since each C These and other objects, features and advantages of the 

component comprises a U and a V value. present invention will be apparent from the following 

Referring to FIG. 2, a conventional video format 4:2:2 is detailed description and the appended claims and drawings 

shown. The video format 4:2:2 has one U,V sample pair for in whidi: 

every pair of Y samples. In the 4:2:2 format of FIG. 2, the ^ FIG. 1 is a 4:2:0 video sampling structure; 

C sampling point is coincident with every other Y sample pjQ 2 is a 4:2:2 video sampling structure; 

point along a raster line. In the 4:2:2 format, the amount of « - . ^ ^ ,^a^ ct^.^.^.- 

. . • J . . J ^- « . - J FIG. 3 IS a 4:4:4 video sampling structure; 

Y component storage required is identical to that required r o . « 

for the C component ^ *s * block diagram of a preferred embodiment of 

Referring to FIG, 3, a conventional video format 4:4:4 is invention; 

shown. The video format 4:4:4 has an identical sample ^ >s a detailed bkxdc diagram of a compoatc with 

structure for the Y, U and V components. In the 4:4:4 format interleave circuit of HG. 4; 

of FIG. 3, the amount of C component storage required is FIG. 6 is a block diagram illustrating an example opera- 

twice thai required for the Y component The 4:4:4 sampling tion of the present invention; 

structure is identical to that used for YUV format graphics ^ FIG. 7 is a block diagram illustrating an example opera- 
data (ix., 24-bit-per-pixel YUV, or 32-bpp aYUV-8888). tion of the present invention; 

^yproaches for conversion between video formats change piG. 8 is a block diagrflm illustrating an example opera- 

the number of chrominance samples C relative to the lumi- ifon of the present invention; 

nance samples Y In particular, to convert video data to rg. 9 is a block diagram illustrating an example opera- 
graphics data, the foUowmg operauons are required, where 55 invention; 
all conversions use the video format 4:4:4 as a common - , ^. .« 1 
conversion step (i.e., conversion from 4:2:0, 4:2:2 or 4:4:4 . >^ ^ ^^"^^ ^'^^"^ illustratmg an example opera- 
video format to 4:4:4 graphics data): ^*>° P^*^°* mvention; and 

(0 scaling and filtering operations in both the horizontal FIG. 11 is a block diagram flluslrating an example opera- 

and vertical directions are necessary for conversion 60 P'^°^ invention. 

fiPom 4:2:0 to 4:4:4. Since norje of the e^g 4:2^0 DETAILED DESCRIPTION OF THE 

chrommance C samphng pomts are ahgped with the PRFFPRRPn FMRODIMENTS 

himinance Y points, it is necessary to inteipolate a PREFERRED EMBODIMENTS 

complete set of C values using either 0.75:0.25 or The implementation of a block move engine (BMQ (a bit 

0.25:0.75 ooefificient pairs for the interpolation; 65 blitter or blitting engine) for rapidly copying blocks of 

(ii) scaling and filtering operations in the horizontal graphics data fiom one location in memory to another is 

direction only is necessary for conversion from 4:2:2 to generally used for graphics processing. BMEs may be 
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exteoded to include two input data streams of identical size, signal FRONT, an input 154a that may receive tbe signal 

which are combined by a logical compositioD operation and BACK and an input 156a that may receive the signals 

written back to memory as a single data block. The demand MASK and ALPHA. Similarly, the circuits 142 and 144 may 

for improvements m graphics ^ed and resolution and tbe ^ respectively, inputs 1506 and 150c that may receive 
convergence of video ^^Pbi^ applications onto com- 5 COMFTYPE. inputs 1526 and 152c that may 

mon platforms has made It desirable to mcorporale a wider . , rmrvCrr - . ic>«L ^ ie>i .u . 

selectionof fonctionswithintbegeneralstriiciireof aBME^ ^^'^^ ^ ^^"^ ^fP'''^ ^ff^J^.^.t^ ""'^ 

The present invention may be configured to allow alteration ^ccive the signal BACK and mputs 1566 and 15fc that may 

of the sampling structure of video chrominance C receive the agnals MASK and ALPHA. 
compoDcnts. ihus pecioitting the convecsioD between video ^ave an output 160 connected to an 

and graphics data fonnals. , ... ia^ -A^ ■ •. ». 

. . - ... J. - . . . mput 162 of the multiplexer 14*. The annul 142 may have 

Refemng to FIG. 4. a block diagram of aarcuit 100 is J cxyoncCtd to an input 166 of the mulUplexer 

shown m accordance with » P«fcned embodiment of the ^ ^ ^ 

present invention. The orout 100 may implement a block ,™k » i. L, tA^ -w^ ,.■ , t*^ 

modify and move engine (BMME) that may provide a ""f"' ?^ the mulUplexer 146. TTk multiptexer 146 

generic framework for integrating video and graphics pro- " °>»y »° "V*" ^^'^V ^ ^ 

cessing The circuit 100 generally comprises a circuit 102. a TYPE- multiplexer 146 may present a signal received 

circuit 104, a dicuit 106 and a memory 108. The drcuit 102 from tbe circuit 140, the circuit 142 or tbe circmt 144 as the 

may be implemented as a color conversion circuit In certain signal RESULT in response to the signal COMPTYPE 
applications, the color conversion circuit 102 may be ^ a preferred embodiment of the present invention, a daU 

^. comiection with HGS. 6. 7. 9 nodificalionsectfon of the BMME 100 may comprise one or 

^m:Tb:^'^V>*^y^m>i^n>catcd3Sisc^and ^ ^ ^ ^^^^ p^,^ ^ ,^ ^ 

filter circuit The circuit 106 may be implemented as a - c -j • u ™ ui-o 

... . ^ , 4^ ^ . convemoo of video to graphics RGB is needed, the color 

composite with mterleave circuit The memory 108 may be • . , . T • i * j T»n. .t- *j 

. , , . * - •* conversion block 102 may be implemented. When the video 

implemented as a system memory. The circuit 100 may ^ ^ . . . * , ^ . i j ct. 

comprise a modified veision of a typical composite section ^rmat 4:4:4 is ttie only foiinat us«i, the sade and filter 
(e.g., the composite and interleave circmt lOQ that may °>»y ^ O"^"*' ^ suitable design for tbe color 

permit interleaving of Y and C component video daU to a conveision block 102 is described m coi)endmg apphcation 

single graphics daU stream. Additionally, the circuit 100 U.S. Ser. No. 09/878,594, filed Jun. 11, 2001, which is 
may allow other functional bk)cks to be inchided to provide „ hereby incorporated by reference in its entirety. A suitable 

unique features required by a particular application. design for tbe scale and filter block 104 maybe described in 

Tbe memory 108 may have an output HO that may co-pending plication VS. Ser. No. 09/960,572, filed Sep. 

present a signal (e.g., FRONT) to an input 112 of the color 21, 2001 and/or Ser. No. 09/878,594, filed Jun. 11, 2001, 

convert circuit 102. The circuit 102 may have an output 114 which are hereby incorporated by reference in their entirety, 
connected tojm input U6 ofAe scale and filter «Acuit 104 ^ operations block 140 of the composite with 

The scale and filter orcuit 104 may have an output 118 that . . °Vt , : , *i-:_iu-*-,' 

1 1 r-n^vi^ * r^-wi r mterlcave block 106 may onplement logical bitwise opera- 

may present a signal (eg., FRONT) to an mput 120 of the ^ j r &• 

circuit 106. The system memory 108 may also have an 
output 122 that may present a signal (e.g., BACK) to an 

input 124 of tbe circuit 106 and an ou^ut 126 that may ^ RESUir-raoOT XOR back 

present a number of signals (e.g., NfASK and ALPHA) to an 

input 128 of the circuit 106. The signals MASK and ALPHA "^^^"^ ^ 

may be presented together or individually. Additionally, ^ 

each of the signals MASK and ALPHA may or may not be ™, , . ^. , <■ 

. . , ^ i ' t * *i 4^ •* The alpha operations block 142 of the composite with 

required for a particular implementation. The circuit 106 . . ^ ...... 

may have an output 130 that may present a signal (e.g., mterkave circmt 106 may perform alpha^lendingequaUons 

RESULT) to an input 132 of the system memory 108. ^ 

The compoate with interleave circuit 106 may also have „„„ , _ „ ^ 

an input 134ibat may i«eive a signal (e.g.. COMFTYPE). RESUiiKAU.HA-FRoyiM(i-ALPHA).BACK)> 

The signal COMFTYPE may be externally generated and 50 ™ . , - , . 

written to a BMME control register by a CPU (both of which ^h^ interleave operations block 144 of the composite with 

are not shown). The signal COMFTYPE may control the interleave circuit 106 generally comprises multiplexers and 

type of data combination operation carried out by the bit shifters (not shown). The interleave operaUons block 144 
composite and interleave block 106. The various signals of ^ay perform (but is not limited to) the foltowing operations: 
the present invention may be implemented as multi-bit or 55 ^^ t^fee daU from the bus FRONT and pass the daU 
single bit signals or busses. unchanged to the bus RESULT, (tbe data may be 

Referring to RG. 5, a detailed block diagram of the chroma (U,V) pairs, or YUV, RGB, aYUV or oRGB 

composite with interleave circuit 106 is shown. The circuit pixels); 

106 generally comprises a circuit 140, a circuit 142, a drcuit /-v.. • m^rvj. r u I-n/^vrf 

I44L a dLit 146. The circuit 14i may be implemented «, ^'^'T'^V^ ^'^ol'^^ 

as a logical operations drcuit (block). J)k drcuit 142 may ^ ^jla^ ^^'^.^^F^ ""^^ T'^^'^V^.u^ 

be implemented as an alpha operations drcuit (block). TTie 24^P YUV values on the bus RESULT; 

circuit 144 may be implemented as an inlerfeave operations and/or 

circuit (block). The circuit 146 may be implemented as a (iii) lake diroma-only (U,V) daU from tbe bus FRONT, Y 
multiplexer. 65 data from the bus BACK and alpha data from the bus 

The drcuit 140 may have an input 150a that may receive ALPHA and combine the data to make up 32-bpp 

the signal COMFTYPE, an input 152fl that may receive the aYUV vahies on the bus RESULT 
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Example operations of Ibe interleave operations block 144 
arc sbowD in the following TABLES la zad lb, 

TABLE la 
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Ibe entries in the TABLE la and the TABLE lb may 
relate to the oonveision operations of the interleave circuit 30 
144 of the composite with interleave circuit 106 (described 
in ccMmection with FIGS. 6-11). Input and output busses 
(eg., the bus FRONT, the bus BACK, the bus MASK/ 
ALPHA and an output interleave bus RESULl) may be 
32-bAs wide. Any color or a^ha component bus (e.g., RGB 
or alpha) may be 8-bits wide. However, other appropriate bit 
widths may be implemented to meet the criteria of a par- 
ticular implementation. Additionally, some input bits may 
not be used in one or more conversion operations. Such 
unused bits may be omitted when applicable. 

Referring to FIGS. 6-11, a variety of particular video data 
to graphics data converdon operations using circuits similar 
to the circuit 100 of FIG. 4. are shown. Referring to FIG. 6, 
& circuit IW is shown illustrating a 4:2:0 to 4:4:4 conver- 
sion. The circuit 100 may be similar to the circuit 100. To 
achieve the necessary conversion steps for producing graph- 
ics foraiat data firom video, the BhflME 100* may: (i) scale 
and filter in both the horizontal and vertical directions on 
chrominance data (the data may be required to be passed 
through the BMME IW twice, once for each filtering 50 
direction), (u) interpolate a complete set of new chromi- 
nance C values, since the existing 4:2:0 chrominance C 
sampling points are tK>t aligned with the luminance Y points, 
(iii) alternately set the filter coefficients to 0.75:0.25 and 
0.25:0.75, and (iv) pass the scaled and filtered U, V samples 
through the composite with interleave block 106' back to the 
bus RESULT unchanged. The color conversion circuit 102' 
may not be used. 

The BMME IW (e.g., video 4:2:0 to graphics 4:4:4 
conversion) configuration may also be implemented for 
conversion of video 4:2:2 to graphics 4:4:4. However, a 
single scaling and filtering operation of chrominance C data 
may be performed in the horizontal direction only and the 
filter coe£5cients may be set to alternate between 1:0 and 
03:03 in order to interpolate a new chrominance U,V pair 
between each existing V, V sample. The color conversion 
circuit 102' is generally not used. 
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Referring to FIG. 7, a circuit 100" is shown illustrating a 
graphics 4:4:4 to graphics YUV converaon. The circuit 100" 
may be similar to the circuit 100. Interleaving of data may 
be performed to combine the separate Y and C data blocks 
into 24-bpp YUV pixels. The chrominance U, V values are 
generally input on the bus FRONT and the hmiinance Y 
value may be input on the bus BACK. The composite with 
interleave block 106" may be set (configured) to perform the 
interleaving operation such that the YUV pixels are placed 
on the bus RESULT The color conversion circuit 102 and 
the scaling and filter circuit 104 are generally not used. 

Referring to FIG. 8, a circuit 100'" is shown illustrating a 
graphics 4:4:4 to graphics RGB conversion. The circuit 
100"* may be ^milar to the circuit 100. For conversion of 
graphics 4:4:4 to graphics RGB, a first step may be similar 
to the graphics 4:4:4 to graphics YUV conversion. A second 
step may be a color conversion process, where the YUV data 
on the bus FRONT may be converted to RGB format and 
passed directly to the bus RESULT. 

Referring to FIG. 9, a circuit 100<^ is shown iUustrating 
a graphics 4:4:4 and alpha to graphics aYUV conversion. 
The circuit 100^^^ may be implemented similarly to the 
circuit 100. 

Referring to FIG. 10, a circuit 100^^ is shown ilhistrating 
a graphics aYUV to graphics oRGB conversion. The circuit 
100^^ may be similar to the circuit 100. The present inven- 
tion may also include a^ha components. The circuit 100^^ 
or the circuit 100^^ may include any interleaving aixi/or 
color conversion process of an alpha channel value. 

Referring to FIG. U, a circuit 100^*^^ is shown illustrating 
a 4:2:2 to graphics YUV conversion. The circuit 100^*^ may 
be similar to the circuit 100. The circuit 100^^ may combine 
filtering and interleaving within a single circuit. The circuit 
100^*^ may reduce the 4:2:2 to graphics YUV operation to a 
single pass of data through the BMME 100<'^. Such a 
configuration may provide a savings in processing time and 
system bus usage. In the case of 4:2:0 to graphics YUV, the 
second filtering pass may be combined with the interleaving 
operation. 

The present invention may provide a video sampling 
structure conversion engine that may permit interieaving of 
luminance and chrominance video data into a single video 
data stream. The single video data stream may be easily 
converted to a graphics data format 

While the invention has been particularly shown and 
described with reference to the preferred embodiments 
thereof, it will be understood by those skilled in the art that 
various changes in form and details may be made without 
departing &om the spirit and scope of the invention. 

What is claimed is: 

1. An apparatus comprising: 

a memory connected to a first input bus, a second iriput 
bus and an ou^ut bus; 

a converter circuit configured to generate a first interme- 
diate signal in response to converting a color format for 
a first data stream received on said first input bus; 

a scale and filter circuit configured to generate a second 
intermediate signal by scaling arxl filtering said first 
intermediate signal; and 

a composite circuit configured to generate an ou^ut 
graphics stream on said output bus by interleaving said 
second intermediate signal and a second data stream 
received on said second input bus. 

2. The apparatus according to claim 1, wherein said 
apparatus comprises a block modify and move engine, 
(BMME) configured to modify a phirahty of blocks firom 
said first data stream while moving said blocks from one 
location to another location in said memory. 
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3. The apparatus according to claim 1, wherein said 
apparatus comprises a video sampling conversion block 
modify and move engine. 

4. The apparatus according to claim 1, wherein said 
apparatus is configured to permit conversion between video 
data and graphics data. 

5. The apparatus according to claim 1, wherein said 
composite circuit comprises: 

a logical operations circuit configured to perform logical 
bitwise operations on said first data stream and said 
second data stream; 

a alpha operations circuit configured to perform alpha- 
blending between said first data stream and said second 
data stream; and 

an interleave operations circuit configured to perform one 
or more predetermmed interleave operations using said 
first data stream. 

6. The apparatus according to claim 5, wherein said one 
or more predetermined interleave operations are selected 
from the group consisting of (i) receive data from said first 
input bus and pass said data unchanged to said ou^ut bus, 
(ii^ receive chrominance data from said first input bus and 
luminance data from said second input bus and combine said 
chrominance and said luminance data to form a first par- 
ticular output value on said output bus, and (iii) receive said 
chrominance data from said first input bus, said luminance 
data from said secortd input bus and alpha data from a third 
input bus and combine said chrominance, said luminance 
and said alpha data to form a second predetermined value on 
said output bus. 

7. The apparatus according to claim 1, further compri^g: 
a memory configiu'ed to interface to said composite circuit 

via a third input bus. 

8. Hie apparatus according to claim 1, wherein said 
apparatus is configured to perform conversion of one or 
more video data formats to graphics data. 

9. The apparatus according to claim 1, fiulher comprising 
a third input bus disposed between said memory and said 
composite circuit to carry a mask signal to control masking 
of said first data stream. 

10. The apparatus according to claim 1, further compris- 
ing a third input bus disposed between said memory and said 
composite circuit to carry an alpha signal to control an 
alpha-blending of said first data stream with said second data 
stream. 

11. The apparatus according to claim 5, wherein said 
composite circuit is further configured to select one of said 
logical operations circuit, said alpha circuit and said inter- 
leave operations circuit to generate said output graphics 
stream in response to a command signal. 

12. An apparatus comprising: 

means for generating a first intermediate signal by modi- 
fying a color format for a first data steam received from 
a memory; 

means for generating a secorxl intermediate signal by 
scaling and filtering said first intermediate signal; 

means for interleaving said second intermediate signal 
and a second data stream received from said memory to 
generate an output graphics stream; and 

means for storing said output graphics stream in said 
memory. 
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13. A method for permitting conversion between video 
data and graphics data, comprising the steps of: 

(A) generating a first inicrmcdiaie signal by modifying a 
color format for a first data steam received from a 

^ memory; 

(B) generating a second intermediate signal by scaling 
and filtering said first intermediate signal; 

(Q interleaving said second intermediate signal and a 
10 second data stream received from said memory to 
generate an output graphics stream and; 
(D) storing said output graphics stream in said memory. 

14. The method according to claim 13; wherein step (C) 
comprises the sub-steps of: 

(C-1) performing logical bitwise operations on said first 
data stream and said second data stream; 

(C-2) performing alpha-blending between said first data 
stream and said second data stream; and 
20 (C-3) performing one or more predetermined interleave 
operations using said first data stream. 

15. The method according to claim 14, wherein sub-step 
(C-3) comprises one or more of the following steps: 

(i) receiving data from a first input bus and passing said 
^ data unchanged to an output bus; 

(ii) receiving chrominance data from said first input bus 
and luminance data from a second input bus and 
combining said chrominance arxl said luminance data 
to form a first particular output value on said output 

^ bus; and 

(iii) receiving said chrominance data from said first input 
bus, said luminance data from said second input bus 
and alpha data from a third input bus and combining 
said chrotmnance, said luminance and said alpha data 
to form a second predetermined vahie on said output 
bus. 

16. The method according to claim 15, further comprising 
the step of: 

40 interfacing said memory with said first input bus and said 
output bus. 

17. The method according to claim 13, further comprising 
the step of: 

converting one or more video data formats in said first 
45 data stream to graphics data in said output graphics 
stream. 

18. The method according to claim 13, wherein step (B) 
comprises the sub-step of filtering said first data stream in a 
first direction and said method further comprises the step of: 

^0 filtering said output graphics stream in a second direction. 

19. The method according to claim 13, wherein step (Q 
comprises the sub-step of: 

generating said output graphics stream by performing an 
exclusive OR operation between said first data stream 
and said second data stream. 

20. The method according to claim 13, wherein step (Q 
comprises the sub-step of: 

generating said output graphics stream by performing a 
^ masking operation on said first data stream and said 
second data stream. 

♦ # * * « 
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ABSTRACT 



A method of forming an image scaling filter for converting 
a first number of input lines to a second number of output 
lines comprises the steps of determining an optimal fre- 
quency response without sharp cutoSis for a given scaling 
factor and for a determined number of taps per line 102. 
Filter coefficients are provided based on the determined filter 
taps per line and the determined optimal frequency response 
103. The coefficients are grouped into sets 104 correspond- 
ing to the taps per output line and the coefficients are 
rescaled so the sum of the set equals one 105. The input lines 
are multiplied by the rescaled coefficients and summed 106 
to achieve the output line values. 

4 Claims, 8 Drawing Sheets 
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METHOD AND APPARATUS FOR FORMING 
IMAGE SCALING FILTERS 

TECHNICAL HELD OF THE INVENTION 

This ioventioD relates to displays and, more particularly, 5 
to the formatioD of image scaling filters for displays. 

BACKGROUND OF THE INVEN^nON 

It has become highly desirable to provide video displays 
in many formats, lliere are different modes for diHerent 
image formats such as panoramic wide screen or movie. 
There are also differeat video formats for computer video 
displays depending upon whether the video display is a VGA 
(640 pixels in each rowx4^0 rows of pixels, where a pixel 
is a picture element), or a super VGA, or an XGA, Further, 
there are different television formats from the standard 
NTSC to as much as 1920 pixels in each row by 1080 rows 
for High Definition TV (HDTV) formal. Further there are 
fiat panel displays and a Digital Mirror Device (DMD), 
which is a new projection display that utilize reflections ^ 
&om hundreds of thousands of micromirrors, each mounted 
above its own cell memory. 

Within the operation of such a system, it may be desirable 
to have a panoramic view or wide screen view, or a movie ^ 
screen view, again requiring some modifications of the 
number of lines of the video signal from a source to the 
number of lines in the output signal. 

This is achieved by some form of image scaling. In order 
to achieve these changes in formats, scaling filters have been ^ 
used. One such known filter is a bilinear interpolation filter, 
which suffers from the two problems of aliasing (unwanted 
patterns) and aperture effect, or image softening (blurring). 
These problems are illustrated in FIG. 1. FIG. la illustrates 
the bilinear filter kernel, FIGS, lb and Ic illustrate the 
frequency response. 

The aliasing problem can be underwood by considering 
the frequency response of the bilinear filter for an example 
of a three to four scaling in FIG. 1. Imag^ scaling is ba^cally 
an image filtering and resampling operation. If X(w) is the 4q 
frequency domain representation of the original signal, then 
the resampling operation will create a scaled image whose 
firequency domain representation contains shifted replicas of 
X(w), located at the new sampling frequency. These replicas 
can interact with each other if they are not completely 45 
separated from each other, leading to aliasing, as shown in 
the hatched portion in FIG. lb. 

The second problem is the aperture effect or image 
softening. This is caused by the fact that interpolation filters 
are basically low pass filters with high frequency roll-off as so 
illustrated by the frequency response of bilinear interpola- 
tion in FIG. Id. 

Doth of these problems relate to the frequency content of 
the signals and can become particularly objectionable with 
high bandwidth sources. 55 

SUMMARY OF THE INVENTION 

In accordance with one embodiment of the present 
invention, an improved image scaling filter is achieved by 
estimating a close to ideal frequency response of the filter 60 
without sharp cut-offs. The estimated frequency response is 
then fed to a design algorithm along with the constraints 
such as a number of filters taps required, which computes a 
set of filler coefficients for scaling. The set of filter coeflS- 
cients are scaled so that each set sums to a value of one. The 65 
appropriate set of input elements is multiplied by the filter 
coefficients and the set is summed to one output hne. 
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DESCRIPTION OF THE INVENTION 

RG. la illustrates the filter kernel of a bilinear interpo- 
lation filter, 

FIGS, lb and Ic illustrate the frequency response of a 
bilinear interpolation filter with 3 to 4 scaling; and 

FIG, Id iUustrates the aperture effect; 

FIG. 2 is an overall block diagram of a video display 
system; 

FIG. 3 is a flow chart of the method according to the 
present invention; 

FIG. 4a illustrates an ideal frequency response for a 3 to 
4 scale filter, 

no. 46 iUustrates an optimal frequency response; 

FIG, 5 iUustrates how the delays and coefficients arc 
operated on to achieve the desired line output in a vertical 
scaling function; 

FIG. 6 iUustrates frequency domain responses of the 
bilinear filter of the prior art (FIG. 6a) and two new scaling 
filters with a first filter with 4 taps (FIG. 6b) and a second 
filler with 6 taps (FIG. 6c). 

FIG. 7a Ulustrates filter kernel coefficients for bilinear 
filter of prior art; 

FIG. 7b iUustrates the first filter coefficients; 

FIG. 7c iUustrates the second filter coefficients; 

RGS. Sa and Sb illustrates 3 to 4 scaling coefficients 
applied using first filter; and 

FIG. 9 illustrates the scaling results of bilinear filter (9fl), 
first filter (9b), and second filter (9c) for 3 to 4 scaling. - 

DETAILED DESCRIPTION OF PREFERRED t 
EMBODIMENTS 

Referring to FIG. 2, there is iUustrated a video display 
system using the image scaling filters in accordance with the 
present invention. A video source 11, which may be from a 
cable, a VCR, or some other source, is applied through the 
A/D (analog-to-digital) converter 13 and the digital repre- 
sentation of the video signal which includes signal decoding 
at signal processor 14 to convert from decoded composite to 
luminance Y and chrominance C (I & Q) signals whether it 
be interlaced or non-interlaced depending u|>on the applica- 
tion. The output Y arxl C is applied through a scaling filter 
15, which in turn may be applied through a color-space 
conversion 16 dirougb a digital to analog (D/A) converter 
16a to a CRT 17, or directly applied from conversion 16 to 
a display such as a digital mirror device (DMD) display 18, 
which was described in a 1987 article by Larry J. Hombeck, 
a Texas Instrument Inc. scientist and described for example 
in the November issue of The Institute of Electrical and 
Electronic Engineers Spectrum, pages 27-31. In some cases, 
the scaling may be done after the color-space conversion. In 
the case of a VGA input, no color-space conversion is used. 
The video signals may be digital, in which case there is no 
analog-to-digital conversion. The input digital signals may 
also be non-video imagery digital signals from a photo- 
graphic or printing device, and the picture elements from 
these are to be scaled up (increasing picture size) or scaled 
down (decreasing picture size). 

Referring to FIG. 3, there is iUustrated a flow chart for the 
method of the present invention for designing optimal 
scaling filters to overcome the problems explained in the 
Background of the Invention Section. The first step 101 is to 
determine the ideal frequency response. FIG. lb iUustrates 
by the heavy line marked "ideal" identifies for the 3 to 4 
scaling the ideal frequency response. This ideal frequency 
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response eliminates aliasmg whfle aUhe same lime reducing algorithm computes the set of multiplier coefficients to 

the aperture effect However, such ideal frequency response match with the number of taps in the optimal frequency 

with abrupt cutoff cannot be realized in practice. response. The coefficients arc ooUcctcd or binned into sets 

™^ ^ ,1. • M * * J A..! for a given output line in step 104. The coefficient sets (step 

FIG. 4a, there is illustrated another ideal frequency l^-v ^ r u . «f th^ 

r ^ . A 1 ■ f .L * c 105) arc rescaled so for each set so the sum oi the coem- 

response for a 3 to 4 scale filter. The zero crossmg of that 5 ^y^/ <^'^ , ,„nii;«i.Vri anH 

. r . if^'Ti. JiiJ^ aents equals one (1). The input unes are muitipliea and 

s«Dal ch^cs for each scale factor. The .deal frequency ^^^^^ 106 to provide Ihc values on the outpullines. 

response, however, has a response that goes on for mfim y ^ J 

a«l also cannot be reahzed m practice In accordance with ^ ^.^ P ^ coefficfents at the four 

the present mvention, for a given scale factor Applicant s , . , a * a-x tti« f«,.r 

. . c . .1. -J I n„ J in multiphers 41 and summed at summer 43. ine tour mputs 

mventton computes the ideal ftequenqr "^PO"-* ^nd 10 ^ ^ 

UKsn uses that m step 1«2 to estunate a desued optunal .^^^J^^^ ^^^^ frequency domain 

frequency response. Compuuuon of optima^ frequency ^ ^j^^ ^ ^ ^^^^^ 

response from ideal frequei.qr 'esponse « an iteraUve pn>- P"^ ^ ^ ^^^^^ ^ ^ Pj,,^^ j 

ces^ Merent numbcR of filter Ups and smooUungs are respectively. Filter 1 (FIG. 6b) has 4 taps per line. whUe 

tn«land mead, aeratK>n the difference beUveen the optunal 15 f^^^^^^^^-^^^^^ TT,, f^^^ .^sponse 

and acnial frequency «sponse k measured. Tins process is J ^ comparison. Ibt 

Iterated umd the opUmal and ideal r^ponse are dose to each ^ corresponds to 

Other (withm acceptable bound of tolerance). The optimal th ali ^ j -r 

frequency response is doc« by smoothing depending upon ncTmrn the filter kernels for bilinear interpola- 

how many fiUer FIG. 46 dlustrat^ the smoothmg 20 ^^^^^ ^ 

esumate of FIG 4a Once this smoothmg esUmate is ^ ^ .^4^^,^ 

determmed, then It is fed mlo a filter design algonthm (such & ' -^^ ,w „w*k .^ffirW*.nfc 

n J II I \ - f im I ™*K polatioQ according to the pnor art with the coeHicients 

as the Paiks^McClellan algorithm) m step 103, along with r„ ^ . ^ Si i.L « ^fT;^«>nte 

. . . , . * ' D xjr^-iK« illustrated at two taps per hne. There are seven coettiaents, 

the desued number of filter taps The Parks-McQeUan and^^enth coefficient being at 0.25, the 

^^^^n ^ y fi^"^^" ^ Pf ^L^^" second and sixth coefficient being at 0.50, the fourth coef- 

MAI^ software pac^^ 10 

tuate Place, 24 Prm^e Parkway, NaUck, Mass. 01760. «^ffieicnts Oiree and five being at 0.75. In accordance with 

The PariES-McQellan FIR filter design algonthm is per- AppUcanl's invention shown in HG. 76. the FUter 1 with 

haps the most popular and widely used FIR filter design ^ y^^ 15 coefficients with the vahies indi- 

methodology. In the Signal Processing Toolbox, the fiinction ^^^^ ranging on both the first and the fifteenth being 

called rcmez designs linear phase HR filters using the _o.o019 and having a maximum value of 0.25 at the eighth 

Parks-Mcaellan algorithm. The Parks-McQellan algorithm coefficient The first coefficient is binned with the fifth, the 

uses the Remez exchange algorithm and Chebyshev ninth, and the thirteenth, which binned set then corresponds 

approximation theory to design filters with optimal fits to a coefficient set for a given output line. HG. 76 illustrates 

between the desired and actual frequency re^nses. The ^j^^ coefficients for four taps per line for a 3 to 4 

filters are optimal in the sense that the maximum error scaUng filter. The number shown in the figure is the output 

between the desired firequency refuse and the actual ^^^^ ^^^^ algorithm. As discussed previously, 

frequency re^nse is minimized. Filters deagned this way y^piicanfs method then decodes these numbers into a 

exhibit an equiripple behavior in their frequency response, ^ ^^^^ suitable for actuaUy implementing scaling filters, 

and hence are sometimes called equiripple filters. ^^jp^ ^ ^^^^^ design algorithm, which consists 

Hie frmction of the following set of numbers [-0.0019, -0.0077, -0.0128, 

0.0,0.0494,0.1327,0.2152,0.25,0.2152,0.1327,0.0494, 

b-remcz(»^/ m) ^.Q, -0.0128, -0.0077, and -0.0019] arc binned into the 

returns row vector b containing the n+1 coefficients of the 45 following sets, each set corresponding to a coefficient set 

order in FIR filter whose frequency-magnitude characteris- (one coefficient for each input line) for an output line: 

tics match those given by vectors f and m. Vectors f and m [-0,0019,0.0494,0.2152,-0.0128], [-0.0077,0.1327, 

specify the frequency-magnitude characteristics of the filter 0.1327,-0.0077], 

f is a vector of frequency points, specified in the range [-0,0128,0.2152,0.0494,-0.0019], [0.0,0.25,0.0]. 

between 0 and 1, where 1.0 corrc^nds to half the 50 Since there are four taps per output line, every fourth 

sample frequency (the Nyquist frequency). coefficient represents the coefficient for the adjacent line, 

m is a vector containing the desired magnitude response 1^*^^^ coefficient sets are then rescaled so that each set 

at the points specified in f. The elements of m must ^ one and the resulUng values are apphed to the 

appear in equal-valued pairs. multipUer 41 as shown m FIG.5 and as shown m FIG 8. 

e l .1. 1 -TL t *u 55 As illustrated in FIG. 7c, there IS illustrated a design filter 

f and ra must be the same length. The length must be an .. .. . .i- -.u . . r . 

^ ^ with six laps per output line with twenty-four different 

even number. n--. - r rk. c » tu^ 

coefficients ranging from 0 to a maximum of 0.2 at the 

The first point of f must be 0 and the last point 1. mid-point of coefficient 12. In that embodiment there would 

The frequencies must be in increasing order. t^ six coefficients (six lines) for a given output line, and 

Duplicate frequency points are allowed, but remez will 50 these coefficients would again be rescaled so that the set 

separate them by 0.1 if they are exactly coincident. sums to 1. These scaled coefficients are the multiplier values 

Note that frequency transitions much faster than 0.1 are in FIG. 5. FIG. 8 shows 3 to 4 scaling realization using Filter 
undesirable because they can cause large amounts of l. 

ripple in the magnitude response. Referring to FIG. 8a, there is illustrated how the sam- 

plot (f,m) can be used to display the filter shape. 65 plings would be applied to the lines in the 3 to 4 scaliiig. If 

The number of filter taps directly impacts the cost of the an input line is directly in line with the output line as line 0 
real-time implementation of the filter. The filter design to 0', then it would be a 1 to 1 relationship. If not. Filter 1 
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wilh 4 taps would have the coefficient of -0.8611 from input 
line 1 to output line 1/ and the coefficients of -7.6xl(r^ 
between input line -1 and output line 1', 0.197 from input 
line 0 to output line 1', and -0.0512 from input line 2 to 
output line 1'. These input line coefficients would sum up to 
the total sum of 1. These would be scaled from the coeffi- 
cients represented by the Filter 1. This would represent for 
example the first set for the first output line, the second set 
would follow the next set of coefficients as binned for the 
second output line, the third output line would be the third 
set of coefficients binned, and the fourth output line be the 
fourth set or bin. Thereafter, it would repeat going back to 
the first set or bin, second, third, and then the fourth set or 
bin for the fifth, sixth, seventh, and eighth output lines. 

no. 8* also shows the 3 to 4 scaling for Filter 1, for 
output line 2' taken from line 0, 1, 2, and 3 with the 
coefficients being -0.0308, for input line 0, 0.05308 for lines 
1 and 2, and -0.0308 for line 3. All of these coefficients 
would be rescaled to sum to 1 and that value would become 
the multipliers for the corresponding input line with the lines 
being appropriately delayed as summed as illustrated in FIG. 
5 to derive the output value for line 2'. Similarly the input to 
output line 3* from input lines 1, 2, 3, and 4 would have the 
coefficients indicated as -0.0512 from line 1, 0.8611 from 
line 2, 0.1977 from line 3, and -7.6x10"^ from line 4. 
Filter in FIG. 7b is: 

[-0.0019, -O.O077, -0.0128, 0.0, 0.0494, 0.1327, 0.2152, 
0.25, 02152, 0.1327, 0.0494, 0.0, -0.0128. -O.OOT/, 
-O.0019] 
This is binned into: 

AH:-0.0019, 0.0494, 0.2152, -0.0128] 

BH:-0.0077, 0.1327, 0.1327, -0.0077] 

CH:-0.0128, 0.2152, 0.0494, -0.0019] 

D4^.0, 0 0.25, 0.0] 
Elements of A sum to: 

-0.0019 +0.0494+02152-0.0128-0-2499 
dividing each element of A by this^ 
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^:-7•6xlo-^ 0.1977, 0.8611, -0.0512] 

FIG. 9 shows the scaling results of bilinear (FIG. 9aX Filter 
1 (FIG. 9b% and 2 (FIG. 9c) for 3 to 4 scaling on a circular « 
zone plate. This is a standard test used to evaluate interpo- 



lation algorithms. Note the significant aliasing in the top 
picture with bilinear interpolation in FIG. 9a. The improve- 
ment with Filter 1 (FIG. 9b) and almost complete elimina- 
tion of aliasing wilh Filter 2 (FIG. 9c| 

The above scaling is vertical scaling by changing the 
number of horizontal lines. This scaling can also be applied 
to width scaling by changing the number of pixels (picture 
elements) per line and using adjacent pixels. FIG. 2 would 
have pixel delays rather than line delays. The steps in FIG. 
3 of starting with an ideal frequency response and convert- 
ing to an optimum frequency response by a smoothing 
estimation would be done. The filter coefficients would then 
be determined based on the number of taps per line. The 
coefficients are then binned and rescaled to equal one. The 
adjacent input pixels are multiplied by the coefficients and 
are summed to get the new output pixel. 

What is claimed is: 

1. A method of forming an optimal image scaling filter for 
converting a first number of input pixels to a second number 
of output pixels comprising the steps of: 

determining an ideal frequency response for a given 
scaling factor; 

determining from an ideal frequency response a smooth- 
ing optimal frequency response with a determined 
number of taps; 

generating filter coefficients based on the determined 
number of filter taps and the optimal frequency 
refuse; 

binning said coefficients into a set corresponding to the 
closest input pixels to said output pixel; 

rescaling said coefficients in said set so that each said set 
sums to one; 

multiplying the closest input pixels to an output pixel 
according to said rescaled coefficients in said set; and 

summing said multiplied input pixels in said set to 
achieve the output pixel value. 

2. The method of claim 1 wherein said generating step 
includes using Parks-McClellan algorithm. 

3. The method of claim 1 wherein said input pixels are on 
adjacent input lines and said output pixel is on a closest 
output line. 

4. The method of claim 3 wherein said generating step 
includes using Parks-McClellan algorithm. 



